Thrasymachus

Karma: 1,371

Thrasymachus Jan 12, 2019, 7:26 PM
15 points
on: Why is so much discussion happening in private Google Docs?
I’m someone who both prefers and practises the ‘status quo’.
My impression is the key feature of this is limited (and author controlled) sharing. (There are other nifty features for things like gdocs—e.g. commenting ‘on a line’ - but this practice predates gdocs). The key benefits for ‘me as author’ are these:
1. I can target the best critics: I usually have a good idea of who is likely to help make my work better. If I broadcast, the mean quality of feedback almost certainly goes down.
2. I can leverage existing relationships: The implicit promise if I send out a draft to someone for feedback is I will engage with their criticism seriously (in contrast, there’s no obligation that I ‘should’ respond to every critical comment on a post I write). This both encourages them to do so, and may help further foster a collegial relationship going forward.
3. I can mess up privately: If what I write makes a critical (or embarrassing) mistake, or could be construed to say something objectionable, I’d prefer this be caught in private rather than my failing being on the public record for as long as there’s an internet archive (or someone inclined to take screen shots). (This community is no stranger to people—insiders or outsiders—publishing mordant criticisms of remarks made ‘off the cuff’ to infer serious faults in the speaker).
I also think the current status quo is a pretty good one from an ecosystem wide perspective too: I think there’s a useful division of labour between ‘early stage’ writings to be refined by a smaller network with lower stakes, and ‘final publications’ which the author implicitly offers an assurance (backed by their reputation) that the work is a valuable contribution to the epistemic commons.
For most work there is a ‘refining’ stage, which is better done by smaller pre-selected networks rather than of authors and critics mutually ‘shouting into the void’ (from the author’s side, there will likely be a fair amount of annoying/irrelevant/rubbish criticism; from a critic’s side, a fair risk your careful remarks will be ignored or brushed off).
Publication seems to be better for polished or refined work, as at this stage a) it hopefully it has fewer mistakes and so generally more valuable to the non-critical reader, b) if there is a key mistake/objection neglected (e.g. because the pre-selected network resulted in an echo chamber) disagreement between (‘steel-manned’) positions registered publicly and hashed seems a useful exercise. (I’m generally a fan of more ‘adversarial’ - or at least ‘adversarial-tolerant’ norms for public discussion for this reason.)
This isn’t perfect, although I don’t see the ‘comments going to waste’ issue as the greatest challenge (one can adapt one’s private comments to a public one to post, although I appreciate this is a costlier route than initially writing the public comment—ultimately, if one finds ones private feedback is repeatedly neglected, one can decline to provide it in the first place).
The biggest one I see is the risk of people who can benefit from a ‘selective high-quality feedback network’ (either contributing useful early stage criticism, having good early stage posts, or both) not being able to enter one. Yet so long as members of existing ones still ‘keep an eye out’ for posts and comments from ‘outsiders’, this does provide a means for such people to build up a reputation to be included in future (i.e. if Alice sees Bob make good remarks etc., she’s more interested in ‘running a draft by him’ next time, or to respond positively if Bob asks her to look something over).
What links here?
- Circling as Cousin to Rationality by Vaniver (Jan 1, 2020, 1:16 AM; 73 points)

Thrasymachus Dec 2, 2018, 5:10 AM
12 points
in reply to: ryan_b’s comment on: Genetically Modified Humans Born (Allegedly)
Once again I plead that when you see that an expert community looks like they don’t know what their doing, it is usually more accurate to ‘reduce confidence’ in your understanding rather than their competence. The questions were patently not ‘about forms’, and covered pretty well the things I would have in mind (I’m a doctor, and I have fairly extensive knowledge of medical ethics).
To explain:
- Although ‘institutional oversight’ in medicine is often derided (IRB creep, regulatory burden, and so on and so forth), one of its main purposes is to act as a check on researchers (whatever their intent) causing harm to their patients, and the idea it is good to have other people besides the researcher (who might be biased) and the patient (who might be less well informed) be the only ones making these decisions. That typical oversight was bypassed here is telling, but perhaps unsurprising as no one would green-light violating a moratorium to subject healthy embryos to poorly tested medical procedures for at best marginal clinical benefit.
- A lot of questions targeted how informed the consent was, because this was often relied upon in the presentation (e.g. “Well, we didn’t get the right mutation, but it was pretty close, and the parents were happy for us to go ahead, so we did”).
- The ‘read and understand’ question (I’m using the transcript, so maybe there were dumber questions which were edited out) wasn’t a question about whether the patients were literate, but whether they had adequate understanding of (e.g.) the technical caveats which they were giving consent to proceed with (e.g. one mutation was a 15 del rather than a 32 del, which rather than the natural mutation which induces a frame shift and the non-functional protein gives a novel protein with a five aa removal, which may still generate an HIV susceptible protein and some remote chance of other biological effects).
- The ‘training’ question is because establishing whether consent is ‘informed’, or providing the necessary information to make it so, isn’t always straightforward (have you ever had a conversation where you thought someone understood you, but later you found out they didn’t?) I did a fair amount of this in medschool, and I don’t think many people think this should be an amateur sport.
- (As hopefully goes without saying, having two rounds of consent where in each the consent taker is a researcher with a vested interest in the work going ahead has obvious problems, and hence why we’re so keen on third party oversight).
- I also see in the transcript fairly extensive discussion about risks (off-target worries would have been tacit knowledge to the audience, so some of this was pre-empted in the presentation then later picked at), and plans for followup etc.

Thrasymachus Nov 5, 2018, 8:35 PM
25 points
on: No Really, Why Aren’t Rationalists Winning?
I don’t see the ‘why aren’t you winning?’ critique as that powerful, and I’m someone who tends critical of rationality writ-large.
High-IQ societies and superforecasters select for demonstrable performance at being smart/epistemically rational. Yet on surveying these groups you see things like, “People generally do better-than-average by commonsense metrics, some are doing great, but it isn’t like everyone is a millionaire”. Given the barrier to entry to the rationalist community is more, “sincere interest” than “top X-percentile of the population”, it would remarkable if they exhibited even better outcomes as a cohort.
There’s also going to be messy causal inference worries that cut either way. If there is in some sense ‘adverse selection’ (perhaps alike IQ societies) for rationalists tending to have less aptitude at social communication, greater prevalence of mental illness (or whatever else), then these people enjoying modest to good success in their lives reflects extremely well on the rationalist community. Contrariwise, there’s plausible confounding where smart creative people will naturally gravitate to rationality-esque discussion, even if this discussion doesn’t improve their effectiveness (I think a lot of non-rationalists were around OB/LW in the early days): the cohort of people who ‘teach themselves general relativity for fun’ may also enjoy much better than average success, but it probably wasn’t the relativity which did it.
A deeper worry wrt to rationality is there may not be anything to be taught. The elements of (say) RQ don’t show much of a common factor (unlike IQ), correlate more strongly with IQ than one another, and improvements in rational thinking have limited domain transfer. So there might not be much of a general sense of (epistemic) rationality, and limited hope for someone to substantially improve themselves in this area.

Thrasymachus Sep 8, 2018, 12:51 PM
19 points
in reply to: ioannes’s comment on: No standard metric for CFAR workshops?
Another thing I’d be particularly interested in is longer term follow-up. It would be impressive if the changes to conscientiousness etc. observed in the 2015 study persist now.

Thrasymachus Sep 6, 2018, 3:08 PM
11 points
in reply to: Davis_Kingsley’s comment on: Great Founder Theory
I’d be hesitant to defend Great Man theory (and so would apply similar caution) but I think it can go some way, especially for defending a fragility of history hypothesis.
In precis (more here):
1. Conception of any given person seems very fragile. If parents decide to conceive an hour earlier or later (or have done different things earlier in the day, etc. etc.), it seems likely another one of the 100 million available sperm fuses than the one which did. The counterpart seems naturally modelled by a sibling, and siblings are considerably different from one another.
2. Although sometimes (/often) supposed Great Men are mere errands of providence, its hard to say this is always the case. It seems the 20th century would have been pretty different if Hitler was not around to rise to power, the character of world religions would be different with siblings of Jesus, Muhammad etc. and Tolstoy’s brother probably wouldn’t have written War and Peace anyway. (Although maybe in some areas ramifications are less pronounced—Great Scientists may alter the timing of discoveries a bit, but it looks plausible that we’d have Relativity by now even without Einstein).
3. 1 and 2 suggests you could get a lot of scrambling of who is around. Even if it was inevitable there was a Mongol expansion, the precise nature of this seems sensitive to who is in charge, and so whether Ghengis Khan or his sibling was born. The precise details of this expansion (where gets encroached on first, which battles are fought, etc. etc.) does horizontally perturb whether, when (and with who) other people conceive children. These different children go on to alter vertically and horizontally who else is conceived, and so the conceptive chaos propagates. I’d semi-seriously defend the thesis that none of us would be here if Ghengis Khan’s parents decided to wait an hour before having sex.
4. This wouldn’t mean the world is merely putty to be sculpted by great men. But even if the stage (and dramatis personae) of history is set by broader factors, which actors take on the role might still have considerable effects on the performance.

Thrasymachus Aug 21, 2018, 11:22 AM
7 points
in reply to: Unnamed’s comment on: Historical mathematicians exhibit a birth order effect too
I not sure t-tests are the best approach to take compared to something non-parametric, given smallish sample, considerable skew, etc. (this paper’s statistical methods section is pretty handy). Nonetheless I’m confident the considerable effect size (in relative terms, almost a doubling) is not an artefact of statistical technique: when I plugged the numbers into a chi-squared calculator I got P < 0.001, and I’m confident a permutation technique or similar would find much the same.

Thrasymachus Jul 3, 2018, 10:49 PM
4 points
in reply to: Thrasymachus’s comment on: Informational hazards and the cost-effectiveness of open discussion of catastrophic risks
0: We agree potentially hazardous information should only be disclosed (or potentially discovered) when the benefits of disclosure (or discovery) outweigh the downsides. Heuristics can make principles concrete, and a rule of thumb I try to follow is to have a clear objective in mind for gathering or disclosing such information (and being wary of vague justifications like ‘improving background knowledge’ or ‘better epistemic commons’) and incur the least possible information hazard in achieving this.
A further heuristic which seems right to me is one should disclose information in the way that maximally disadvantages bad actors versus good ones. There are a wide spectrum of approaches that could be taken that lie between ‘try to forget about it’, and ‘broadcast publicly’, and I think one of the intermediate options is often best.
1: I disagree with many of the considerations which push towards more open disclosure and discussion.
1.1: I don’t think we should be confident there is little downside in disclosing dangers a sophisticated bad actor would likely rediscover themselves. Not all plausible bad actors are sophisticated: a typical criminal or terrorist is no mastermind, and so may not make (to us) relatively straightforward insights, but could still ‘pick them up’ from elsewhere.
1.2: Although a big fan of epistemic modesty (and generally a detractor of ‘EA exceptionalism’), EAs do have an impressive track record in coming up with novel and important ideas. So there is some chance of coming up with something novel and dangerous even without exceptional effort.
1.3: I emphatically disagree we are at ‘infohazard saturation’ where the situation re. Infohazards ‘can’t get any worse’. I also find it unfathomable ever being confident enough in this claim to base strategy upon its assumption (cf. eukaryote’s comment).
1.4: There are some benefits to getting out ‘in front’ of more reckless disclosure by someone else. Yet in cases where one wouldn’t want to disclose it oneself, delaying the downsides of wide disclosure as long as possible seems usually more important, and so rules against bringing this to an end by disclosing yourself save in (rare) cases one knows disclosure is imminent rather than merely possible.
2: I don’t think there’s a neat distinction between ‘technical dangerous information’ and ‘broader ideas about possible risks’, with the latter being generally safe to publicise and discuss.
2.1: It seems easy to imagine cases where the general idea comprises most of the danger. The conceptual step to a ‘key insight’ of how something could be dangerously misused ‘in principle’ might be much harder to make than subsequent steps from this insight to realising this danger ‘in practice’. In such cases the insight is the key bottleneck for bad actors traversing the risk pipeline, and so comprises a major information hazard.
2.2: For similar reasons, highlighting a neglected-by-public-discussion part of the risk landscape where one suspects information hazards lie has a considerable downside, as increased attention could prompt investigation which brings these currently dormant hazards to light.
3: Even if I take the downside risks as weightier than you, one still needs to weigh these against the benefits. I take the benefit of ‘general (or public) disclosure’ to have little marginal benefit above more limited disclosure targeted to key stakeholders. As the latter approach greatly reduces the downside risks, this is usually the better strategy by the lights of cost/benefit. At least trying targeted disclosure first seems a robustly better strategy than skipping straight to public discussion (cf.).
3.1: In bio (and I think elsewhere) the set of people who are relevant setting strategy and otherwise contributing to reducing a given risk is usually small and known (e.g. particular academics, parts of the government, civil society, and so on). A particular scientist unwittingly performing research with misuse potential might need to know the risks of their work (likewise some relevant policy and security stakeholders), but the added upside to illustrating these risks in the scientific literature is limited (and the added downsides much greater). The upside of discussing them in the popular/generalist literature (including EA literature not narrowly targeted at those working on biorisk) is limited still further.
3.2: Information also informs decisions around how to weigh causes relative to one another. Yet less-hazardous information (e.g. the basic motivation given here or here, and you could throw in social epistemic steers from the prevailing views of EA ‘cognoscenti’) is sufficient for most decisions and decision-makers. The cases where this nonetheless might be ‘worth it’ (e.g. you are a decision maker allocating a large pool of human or monetary capital between cause areas) are few and so targeted disclosure (similar to 3.1 above) looks better.
3.3: Beyond the direct cost of potentially giving bad actors good ideas, the benefits of more public discussion may not be very high. There are many ways public discussion could be counter-productive (e.g. alarmism, ill-advised remarks poisoning our relationship with scientific groups, etc.). I’d suggest the examples of cryonics, AI safety, GMOs and other lowlights of public communication of policy and science are relevant cautionary examples.
4: I also want to supply other more general considerations which point towards a very high degree caution:
4.1: In addition to the considerations around the unilateralist’s curse offered by Brian Wang (I have written a bit about this in the context of biotechnology here) there is also an asymmetry in the sense that it is much easier to disclose previously-secret information than make previously-disclosed information secret. The irreversibility of disclosure warrants further caution in cases of uncertainty like this.
4.2: I take the examples of analogous fields to also support great caution. As you note, there is a norm in computer security of ‘don’t publicise a vulnerability until there’s a fix in place’, and initially informing a responsible party to give them the opportunity to to do this pre-publication. Applied to bio, this suggests targeted disclosure to those best placed to mitigate the information hazard, rather than public discussion in the hopes of prompting a fix to be produced. (Not to mention a ‘fix’ in this area might prove much more challenging than pushing a software update.)
4.3: More distantly, adversarial work (e.g. red-teaming exercises) is usually done by professionals, with a concrete decision-relevant objective in mind, with exceptional care paid to operational security, and their results are seldom made publicly available. This is for exercises which generate information hazards for a particular group or organisation—similar or greater caution should apply to exercises that one anticipates could generate information hazardous for everyone.
4.4: Even more distantly, norms of intellectual openness are used more in some areas, and much less in others (compare the research performed in academia to security services). In areas like bio, the fact that a significant proportion of the risk arises from deliberate misuse by malicious actors means security services seem to provide the closer analogy, and ‘public/open discussion’ is seldom found desirable in these contexts.
5: In my work, I try to approach potentially hazardous areas as obliquely as possible, more along the lines of general considerations of the risk landscape or from the perspective of safety-enhancing technologies and countermeasures. I do basically no ‘red-teamy’ types of research (e.g. brainstorm the nastiest things I can think of, figure out the ‘best’ ways of defeating existing protections, etc.)
(Concretely, this would comprise asking questions like, “How are disease surveillance systems forecast to improve over the medium term, and are there any robustly beneficial characteristics for preventing high-consequence events that can be pushed for?” or “Are there relevant limits which give insight to whether surveillance will be a key plank of the ‘next-gen biosecurity’ portfolio?”, and not things like, “What are the most effective approaches to make pathogen X maximally damaging yet minimally detectable?”)
I expect a non-professional doing more red-teamy work would generate less upside (e.g. less well networked to people who may be in a position to mitigate vulnerabilities they discover, less likely to unwittingly duplicate work) and more downside (e.g. less experience with trying to manage info-hazards well) than I. Given I think this work is usually a bad idea for me to do, I think it’s definitely a bad idea for non-professionals to try.
I therefore hope people working independently on this topic approach ‘object level’ work here with similar aversion to more ‘red-teamy’ stuff, or instead focus on improving their capital by gaining credentials/experience/etc. (this has other benefits: a lot of the best levers in biorisk are working with/alongside existing stakeholders rather than striking out on one’s own, and it’s hard to get a role without (e.g.) graduate training in a relevant field). I hope to produce a list of self-contained projects to help direct laudable ‘EA energy’ to the best ends.

Thrasymachus Jul 3, 2018, 10:48 PM
15 points
on: Informational hazards and the cost-effectiveness of open discussion of catastrophic risks
Thanks for writing this. How best to manage hazardous information is fraught, and although I have some work in draft and under review, much remains unclear—as you say, almost anything could have some some downside risk, and never discussing anything seems a poor approach.
Yet I strongly disagree with the conclusion that the default should be to discuss potentially hazardous (but non-technical) information publicly, and I think your proposals of how to manage these dangers (e.g. talk to one scientist first) generally err too lax. I provide the substance of this disagreement in a child comment.
I’d strongly endorse a heuristic along the lines of, “Try to avoid coming up with (and don’t publish) things which are novel and potentially dangerous”, with the standard of novelty being a relatively uninformed bad actor rather than an expert (e.g. highlighting/elaborating something dangerous which can be found buried in the scientific literature should be avoided).
This expressly includes more general information as well as particular technical points (e.g. “No one seems to be talking about technology X, but here’s why it has really dangerous misuse potential” would ‘count’, even if a particular ‘worked example’ wasn’t included).
I agree it would be good to have direct channels of communication for people considering things like this to get advice on whether projects they have in mind are wise to pursue, and to communicate concerns they have without feeling they need to resort to internet broadcast (cf. Jan Kulveit’s remark).
To these ends, people with concerns/questions of this nature are warmly welcomed and encouraged to contact me to arrange further discussion.

Thrasymachus May 26, 2018, 1:54 PM
8 points
on: Societal Growth Requires Rehabilitation
This seems right to me, and at least the ‘motte’ version of growth mindset accepts that innate ability may set pretty hard envelopes on what you can accomplish regardless of how energetic/agently you pursue self improvement (and this can apply across a range of ability—although it seems cruel and ludicrous to suggest someone with severe cognitive impairment can master calculus, it also seems misguided to suggest someone in middle age can become a sports star if they really go for it). As you say, taking growth mindset ‘too far’ has a dark side in that we might start thinking that people fail or struggle because they aren’t trying hard enough (generally a fault which we morally criticise) rather than lacking the ability (generally ‘blameless’).
But I’d venture a broader criticism about growth mindset which apply both to the ‘motte’ form sketched above, but also its sincere use in the rationalist community—that we shouldn’t only ‘not take it too far’, but not take it anywhere at all:
1) Growth mindset as expounded by Dweck and colleagues has not weathered replication well. The most recent systematic reviews give extremely minor effects on achievement (r=0.1) and even smaller intervention effects (d=0.08). The authors of the meta-analysis are about as sceptical as I am about whether these residual effects are real, but even if real they are extremely minor compared to more traity things (you can get much better prediction of academic achievement by genotyping than assessing growth mindest).
2) There’s a natural story of reverse causation, which also applies to the closely related ‘internal versus external locus of control’. If you’re smart and living in propitious circumstances, you may be right in thinking “I can get good at this if I really try” for many different things. If you lack this good fortune, your belief “Even if I try really hard at something, for most somethings I probably won’t develop mastery (or even competence)” might be a case of accurate and laudable insight.
3a) I can think of more than a few occasions in my life where the latter was better for me. One was when I was contemplating what subjects to keep before I went to university, and I had discussions with various teachers along the lines of, “You’re good but not exceptional at this, maybe think about something else?” Or (in medical school) a conversation along the lines of, “You have dysgraphia, which probably makes you somewhat weaker at fine manual dexterity. Opthalmology requires really good fine manual dexterity, so maybe this isn’t the specialty for you.”
3b) It also seemed to serve me better when I couldn’t circumvent my limitations by picking a different line of work. I focused especially hard on training myself to perform practical procedures because I realised I was working from a disadvantage, and so had to try harder to be satisfactory (I maintained no illusions of becoming great at it).
3c) My impression is that conversations (or thinking) like this tend to be more emotionally difficult than more aspirational, “Don’t worry, you can do it!” exhortations. So I’d guess they are probably under-supplied from their optimum.
In essence, there’s an underlying empirical topic which ‘growth mindset’ relies upon: that a lot of whether one accomplishes something or not depends on mindset or attitude. The answer to that, as best as I can tell, is this isn’t really true: we live in a world which has the uncomfortable features where which tickets one draws from the genetic lottery, birthplace lottery, and early environment lottery (etc) determine the broad strokes of one’s life far more than particular efforts of will and mindset (and growth mindset in particular, which seems to have slim-to-no effect). Many things which are possible for someone are not possible for us, no matter what we do, and no matter how hard we try.
Then there’s a prudential question of (even if it isn’t true) whether it would be better to act and believe as-a growth mindset would suggest. Again, it doesn’t seem so: the evidence for mindset interventions working is slim to none, and insofar as one can survey anecdata my impression is ‘anti-growth mindset’ advice is undersupplied relative to its importance.
It is inarguable one should often persevere in work to improve oneself, to not give up ‘too soon’, and to encourage others when trying to do the same. Yet there are times when it is better to recognise the limits of one’s abilities, that one should cut their losses, and to shoulder the burden of (if one believes it to be the case) telling someone they should quit something because they ‘don’t have what it takes’. The right judgement in these cases is a matter for practical wisdom. Insofar as growth mindset as it is preached (but also as it is practised) biases us more to the former sort of behaviour, it should be resisted.

Thrasymachus May 22, 2018, 9:23 AM
21 points
in reply to: Jan_Kulveit’s comment on: The Berkeley Community & The Rest Of Us: A Response to Zvi & Benquo
A healthy topology of the field should have approximately power-law distribution of hub sizes. This should be true also for related research fields we are trying to advance, like AI alignment or x-risk. If the structure is very far from that (e.g. one or two very big hubs, than nothing, than a lot of two orders of magnitude smaller groups fighting for mere existence), the movement should try to re-balance, supporting growth of medium-tier hubs.
Although my understanding of network science is abecedarian, I’m unsure of both whether this feature is diagnostic (i.e. divergence from power-law distributions should be a warning sign) or whether we in fact observe overdispersion even relative to a power law. The latter first.
1) ‘One or two big hubs, then lots of very small groups’ is close to what a power law distribution should look like. If anything, it’s plausible the current topology doesn’t look power-lawy enough. The EA community overlaps with the rationalist community, and it has somewhat better data on topology: If anything the hub sizes of the EA community are pretty even. This also agrees with my impression: although the bay area can be identified as the biggest EA hub, there are similar or at least middle sized hubs elsewhere (Oxford, Cambridge (UK), London, Seattle, Berlin, Geneva, etc. etc.) If we really thought a power law topology was desirable, there’s a plausible case to push for centralisation.
The closest I could find to a ‘rationalist survey’ was the SSC survey, which again has a pretty ‘full middle’, and not one or two groups ascendant. That said, I’d probably defer to others impressions here as I’m not really a rationalist and most of the rationalist online activity I see does originate from the bay. But even if so, this impression wouldn’t worry us if we wanted to see a power law here.
2) My understanding is there are a few generators of power law distributions. One is increasing returns to scale (e.g. cities being more attractive to live in the larger they are, ceteris paribus), another is imperfect substitution (why listen to an okay pianist when I can have a recording of the world’s best?), a third could be positive feedback loops or Matthew effects (maybe ‘getting lucky’ with a breakout single increases my chance of getting noticed again, even when controlling for musical ability versus the hitless).
There are others, but many of these generators are neutral, and some should be welcomed. If there’s increasing marginal returns to rationalist density, inward migration to central hubs seems desirable. Certain ‘jobs’ seem to have this property: a technical AI researcher in (say) Japan probably can have greater EV working in an existing group (most of which are in the bay) rather than trying to seed a new AI safety group in Japan. Ditto if the best people in a smaller hub migrate to contribute to a larger one (although emotions run high, I don’t think calling this ‘raiding’ is helpful—the people who migrate have agency).
[3) My hunch is what might be going on is that the ‘returns’ are sigmoid, and so are diminishing with new entrants to the Bay Area. ‘Jobs’-wise, it is not clear the Bay Area is the best place to go if you aren’t going to work on AI research, and even if so this is a skill set that is rare in absolute terms amongst rationalists). Social-wise, there’s limited interaction bandwidth, especially among higher status folks, and so the typical rationalist who goes to the bay won’t get the upside the most desirable bits of bay area social interactions—when weighed across from the transaction costs, staying put and fostering another hub might look better.]
(I echo Chris’s exhortation)

Thrasymachus Mar 21, 2018, 9:55 PM
15 points
in reply to: ozymandias’s comment on: Dragon Army Retrospective
+1
It also risks a backfire effect. If one is in essence a troll happy to sneer at what rationalists do regardless of merit (e.g. “LOL, look at those losers trying to LARP enders game!”), seeing things like Duncan’s snarky parenthetical remarks would just spur me on, as it implies I’m successfully ‘getting a rise’ out of the target of my abuse.
It seems responses to criticism that is unpleasant or uncharitable are best addressed specifically to the offending remarks (if they’re on LW2, this seems like pointing out the fallacies/downvoting as appropriate), or just ignored. More broadcasted admonishment (“I know this doesn’t apply to everyone, but there’s this minority who said stupid things about this”) seems unlikely to marshall a corps of people who will act together to defend conversational norms, but bickering and uncertainty about whether or not one is included in this ‘bad fraction’.
(For similar reasons, I think amplifying rebuttals along the lines of, “You’re misinterpreting me, and that people who don’t interpret others correctly is one of the key problems with the LW community” seems apt to go poorly—few want to be painted as barbarians at the gates, and prompts those otherwise inclined to admit their mistake to instead double down or argue the case further.)

Thrasymachus Mar 20, 2018, 9:34 PM
10 points
in reply to: adifferentface’s comment on: Dragon Army Retrospective
I also think I got things about right, but I think anyone else taking an outside view would’ve expected roughly the same thing.
I think you might be doing yourself a disservice. I took the majority of contemporary critcism was more directed towards (in caricature) ‘this is going to turn into a nasty cult!’ than (what I took your key insight to be) ‘it will peter out because the commander won’t actually have the required authority’.
So perhaps the typical ‘anyone else’ would have alighted on the wrong outside view, or at least the wrong question to apply it to (‘How likely would a group of rationalists end up sustaining a ‘dictator’ structure?‘, rather than ‘Conditional on having this structure, what would happen next?’)

Thrasymachus Mar 20, 2018, 9:16 PM
1 point
in reply to: adifferentface’s comment on: Dragon Army Retrospective
Bravo—I didn’t look at the initial discussion, or I would have linked your pretty accurate looking analysis (on re-skimming, Deluks also had points along similar lines). My ex ante scepticism was more a general sense than a precise pre-mortem I had in mind.

Thrasymachus Mar 19, 2018, 2:59 PM
32 points
on: Dragon Army Retrospective
Although I was sufficiently sceptical of this idea to doubt it was ‘worth a shot’ ex ante,(1) I was looking forward to being pleasantly surprised ex post. I’m sorry to hear it didn’t turn out as well as hoped. This careful and candid write-up should definitely be included on the ‘plus’ side of the ledger for this project.
With the twin benefits of no skin in the game and hindsight. I’d like to float another account which may synthesize a large part of ‘why it didn’t work’.
Although I understand DAB wasn’t meant to simply emulate ‘military style’ living, it did borrow quite a lot of that framing (hence the ‘army’ and ‘barracks’ bit of DAB). Yet these arrangements require a considerable difference in power and authority between the commander and their subordinates. I don’t think DAB had this, and I suggest this proved its downfall.
[R]ight, as the world goes, is only in question between equals in power, while the strong do what they can and the weak suffer what they must
Thucydides—The Melian Dialogue
First, power. Having a compelling or else matters for maintaining discipline—even if people come in planning to obey, they may vacillate. In military contexts, accomodation at a barracks (or a boot camp) is a feature that can be unilaterally rescinded—if I argue back against the drill instructor, even if I’m in the right, they can make the credible threat “Either do as I say, or I kick you out”. This threat has great ‘downside-asymmetry’: it’s little skin off the instructor’s nose if he boots me, but it means the end of my military career if they follow through. In consequence, the instructor has a lot more de facto bargaining power to make me do things I don’t want to do. In negotiation theory, the drill sergeant’s BATNA is way better than mine, and so good the only ‘negotiation’ they use are threats rapidly escalating to expulsion for any disobedience to their dictation.
I assume the legal ‘fact on the ground’ is that the participants of DAB were co-signatories on a lease, making significant financial contributions, with no mechanism for the designated ‘commander’ to kick people out unilaterally. Given this realpolitic, whatever ‘command’ the commander has in this situation is essentially roleplay. If a ‘Red Knight’ character decides not to play ball with the features of DAB that go beyond typical legal arrangements of house-sharing (e.g. “Nah, I’m not going to do this regular communal activity, I’m not going to do press-ups because you’re telling me to, and I’m going to do my own thing instead.”), the commander can’t ‘expel’ them, but at most ask them to leave.
In fact the commander can only credibly threaten to expel subordinates from the social game of Dragon Army. If I’m a participant this might be a shame, but it might not be a great cost if I’m not feeling it anymore: I can probably (modulo some pretty dodgy social ostracising which other participants may not abide by) still interact with the remaining dragons in a normal housemate-y way.
Such a situation has pretty symmetrical downsides for both parties: if I’m the Red Knight, I probably would end up leaving (it wouldn’t be fun being the ‘odd man out’, although the difficulty is shared by the dragons who have people in the house who aren’t ‘buying in’, and the commander dealing with the ever-present example of the pretty mild consequence of what happens if others likewise disobey), but not until I’d lined up better accomodation, and I certainly wouldn’t feel any obligation—given you’re kicking me out rather than I’m choosing to leave—to pay rent after I go or help find others to ‘fill my spot’ to defray the proportionally increased cost of the lease (itself an incentive for other subordinates to pressure the commander to relent).
Instead of the boot camp case, where the much inferior BATNA of recruits gives them no negotiation position with their drill sergeant; in the DA case, the similarly equidistant BATNAs mean the commander and a dragon are at near-party (whatever affectations both maintain to the contrary). The stage is set for negotiation over discipline (even if coded or implicit), rather than dictation from the commander to the subordinate.
Authority (in the informal or moral sense, c.f. Weberian ‘charismatic authority’) can partly—though imperfectly—substitute for power (e.g. doctors and clergy tend to have more ‘influence’ than ‘ordering people around’). What is notable that authority like this tends to be linked to some particular concrete achievement or track record, and is mainly constrained to a particular domain, with only mild ‘bleed through’ into general social status.
Given the nature of DAB, there is no convincing concrete thing a putative-commander could point to which legitimizes broad-raging authority over others beyond assent by the putative-subordinates this is an experiment they want to try. I suspect this results in pseudo-authority and obedience highly conditional on results: “I’m going to act as-if I am much lower status than this ‘commander’, and so kowtow towards them. But in terms of real status they’re a peer to me, and I spot them this loan of social game status because I may benefit. If this doesn’t happen, I’m less inclined to play along.” The limited capital this supplies doesn’t give the commander much to spend on pushing things through despite dissent: as the matters are between equals, ‘right’ becomes the question again.
I suggest this may explain the need for a better off-ramp (I note military or religious organisations which emphasize discipline and obedience generally provide accomodation etc. gratis, perhaps in part to provide the power initiates are there by invitation—I wonder whether reciprocal arrangements with another house substitute adequately); the reluctance to ‘give enough orders’ (better to maintain a smaller stake of authority than gamble a wider bid that could lose lots of face if flouted); the lack of a ‘big enough stick’ (“I’m happy to undergo token punishment—to lose pretend status—as a ritual for the sake of the game, but if the price of you ‘disciplining’ me costs real status, I’m out”); the need to avoid transgressing what less bought-in subordinates wanted (perhaps the result of a coded and implicit negotiation suggested earlier); and the shortfall between the ostensible standards asserted (set by aspiration), and those observed (set by the balance of power). One could imagine recognition of these features may have been key in transforming ‘White Knights’ into ‘Black Knights’.
I’d guess, if this is the (forgive me) crux of the problem, whether it can really be fixed. Perhaps even more assiduous selection could do the trick, yet the base rate in the Bay Area (which isn’t renowned for selecting people easygoing with authority) is not propitious. An alternative—but an exorbitantly expensive one—which would supply the commander with unquestioned power/authority would be if rent is free for subordinates: “You’re here for free, so I can kick you if you don’t play ball”. I suspect a more modest ‘Phase II’ (maybe just social norms to do communal activity, with no pretended ‘boss’) might be the best realistic iteration.
-
1: It also seemed a bit risky from an ‘abuse’ (very broadly construed) perspective to me, although I stress this worry arose solely from the set-up rather than any adverse judgement of Duncan personally. Although understandable to be snarky to critics (e.g. one despicable pseudonymous commenter on the original ‘pitch’ recommended Duncan kill himself) I think the triumphalism of, “Although it didn’t work, this ‘not working’ was not achieving all the good targets, rather than something horrible happening to someone—guess this wasn’t a hotbed of abuse after all, and I’m not some abusive monster like you said! Eat humble pie! (but you won’t, because you’re irrational!)” is a bit misplaced.
The steelman of this is something like, “Although I don’t think it blowing up really badly is more likely than not, the disjunction of bad outcomes along these lines is substantially higher than the base rate for a typical group-house, and this isn’t outweighed across the scales by the expected benefit”. On this view the observation the proposed risk wasn’t realised is only mild discomfirmation for the hazard, although it rules strongly against outlandish ‘disaster by default’ views.
[Edit: Slight rewording, concision, and I remembered the theory I was gesturing at imprecisely]

Thrasymachus Mar 7, 2018, 3:44 PM
2 points
on: Comments on Power Law Distribution of Individual Impact
This new paper may be of relevance (H/T Steve Hsu). The abstract:
The largely dominant meritocratic paradigm of highly competitive Western cultures is rooted on the belief that success is due mainly, if not exclusively, to personal qualities such as talent, intelligence, skills, efforts or risk taking. Sometimes, we are willing to admit that a certain degree of luck could also play a role in achieving significant material success. But, as a matter of fact, it is rather common to underestimate the importance of external forces in individual successful stories. It is very well known that intelligence or talent exhibit a Gaussian distribution among the population, whereas the distribution of wealth—considered a proxy of success—follows typically a power law (Pareto law). Such a discrepancy between a Normal distribution of inputs, with a typical scale, and the scale invariant distribution of outputs, suggests that some hidden ingredient is at work behind the scenes. In this paper, with the help of a very simple agent-based model, we suggest that such an ingredient is just randomness. In particular, we show that, if it is true that some degree of talent is necessary to be successful in life, almost never the most talented people reach the highest peaks of success, being overtaken by mediocre but sensibly luckier individuals. As to our knowledge, this counterintuitive result—although implicitly suggested between the lines in a vast literature—is quantified here for the first time. It sheds new light on the effectiveness of assessing merit on the basis of the reached level of success and underlines the risks of distributing excessive honors or resources to people who, at the end of the day, could have been simply luckier than others. With the help of this model, several policy hypotheses are also addressed and compared to show the most efficient strategies for public funding of research in order to improve meritocracy, diversity and innovation.

Thrasymachus Feb 25, 2018, 7:57 PM
5 points
in reply to: Qiaochu_Yuan’s comment on: Meta-tations on Moderation: Towards Public Archipelago
I endorse Said’s view, and I’ve written a couple of frontpage posts.
I also add that I think Said is a particularly able and shrewd critic, and I think LW2 would be much poorer if there was a chilling effect on his contributions.

Thrasymachus Feb 18, 2018, 11:45 PM
2 points
in reply to: Ben Pace’s comment on: [Meta] New moderation tools and moderation guidelines
Let’s focus on the substance, please.

Thrasymachus Feb 18, 2018, 4:25 PM
17 points
in reply to: Said Achmiz’s comment on: [Meta] New moderation tools and moderation guidelines
I’m also mystified at why traceless deletition/banning are desirable properties to have on a forum like this. But (with apologies to the moderators) I think consulting the realpolitik will spare us the futile task of litigating these issues on the merits. Consider it instead a fait accompli with the objective to attract a particular writer LW2 wants by catering to his whims.
For whatever reason, Eliezer Yudkowsky wants to have the ability to block commenters and have the ability to do traceless deletion on his own work, and he’s been quite clear this is a condition for his participation. Lo and behold precisely these features have been introduced, with suspiciously convenient karma thresholds which allow EY (at his current karma level) to traceless delete/ban on his own promoted posts, yet exclude (as far as I can tell) the great majority of other writers with curated/front page posts from being able to do the same.
Given the popularity of EY’s writing (and LW2 wants to include future work of his), the LW2 team are obliged to weigh up the (likely detrimental) addition of these features versus the likely positives of his future posts. Going for the latter is probably the right judgement call to make, but let’s not pretend it is a principled one: we are, as the old saw goes, just haggling over the price.

Thrasymachus Jan 21, 2018, 11:26 PM
3 points
on: How the LW2.0 front page could be better at incentivizing good content
FWIW, I struggle to navigate the front page to look at good posts (I struggle to explain why—I think I found ‘frontpage etc.’ easier for earlier versions). What I do instead is look at the comments feed and click through to articles that way, which seems suboptimal, as lots of comments may not be a very precise indicator of quality.

Thrasymachus Jan 21, 2018, 2:37 AM
13 points
in reply to: Said Achmiz’s comment on: Kenshō
FWIW, this aptly describes my own adverse reaction to the OP. “I have this great insight, but I not only can’t explain it to you, but I’m going to spend the balance of my time explaining why you couldn’t understand it if I tried to explain it” sounds awfully close to bulveristic stories like, “If only you weren’t blinded by sin, you too would see the glory of the coming of the lord”.
That the object level benefits offered seem to be idiographic self-exhaltations augur still poorer (i.e. I cut through confusion so much more easily now (no examples provided); I have much greater reserves to do stuff; I can form much deeper pacts with others who, like I, can See the Truth.) I recall the ‘case’ for Ander’s Connection Theory was of a similar type. But at least connection theory at least sketched something like a theory to consider on its merits.
There needs to be either some object-level description (i.e. “This is what Looking is”), or—if that really isn’t possible—demonstration of good results (i.e. “Here’s a great post on a CFAR-adjecent topic, and this was thanks to Looking.”) Otherwise, the recondite and the obscurantist look very much alike.
What links here?
- The three existing ways of explaining the three characteristics of existence by Kaj_Sotala (Mar 7, 2021, 6:20 PM; 40 points)
- Kaj_Sotala's comment on A non-mystical explanation of insight meditation and the three characteristics of existence: introduction and preamble by Kaj_Sotala (May 7, 2020, 11:01 AM; 8 points)