LessWrong developer, rationalist since the Overcoming Bias days. Jargon connoisseur.
jimrandomh
Karma: 21,463
This post used the RSS automatic crossposting feature, which doesn’t currently understand Substack’s footnotes. So, this would require editing it after-crossposting.
I think you’re significantly mistaken about how religion works in practice, and as a result you’re mismodeling what would happen if you tried to apply the same tricks to an LLM.
Religion works by damaging its adherents’ epistemology, in ways that damage their ability to figure out what’s true. They do this because any adherents who are good at figuring out what’s true inevitably deconvert, so there’s both an incentive to prevent good reasoning, and a selection effect where only bad reasoners remain.
And they don’t even succeed at constraining their adherents’ values, or being stable! Deconversion is not rare; it is especially common among people exposed to ideas outside the distribution that the religion built defenses against. And people acting against their religions’ stated values is also not rare; I’m not sure the effect of religion on values-adherence is even a positive correlation.
That doesn’t necessarily mean that there aren’t ideas to be scavenged from religion, but this is definitely salvage epistemology with all the problems that brings.
requiring laborious motions to do the bare minimum of scrubbing required to make society not mad at you
Society has no idea how much scrubbing you do while in the shower. This part is entirely optional.
We don’t yet have collapsible sections in Markdown, but will have them in the next deploy. The syntax will be:
+++ Title Contents More contents +++
I suspect an issue with the RSS cross-posting feature. I think you may used the “Resync RSS” button (possibly to sync an unrelated edit), and that may have fixed it? The logs I’m looking at are consistent with that being what happened.
They were in a kind of janky half-finished state before (only usable in posts not in comments, only usable from an icon in the toolbar rather than the <details> section); writing this policy reminded us to polish it up.
The bar for Quick Takes content is less strict, but the principle that there must be a human portion that meets the bar is the same.
In theory, maybe. In practice, people who can’t write well usually can’t discern well either, and the LLM submissions that are actually submitted to LW have much lower average quality than the human-written posts. Even if they were of similar quality, they’re still drawn from a different distribution, and the LLM-distribution is one that most readers can draw from if they want (with prompts that are customized to what they want), while human-written content is comparatively scarce.
Policy for LLM Writing on LessWrong
This seems like an argument that proves too much; ie, the same argument applies equally to childhood education programs, improving nutrition, etc. The main reason it doesn’t work is that genetic engineering for health and intelligence is mostly positive-sum, not zero-sum. Ie, if people in one (rich) country use genetic engineering to make their descendents smarter and the people in another (poor) country don’t, this seems pretty similar to what has already happened with rich countries investing in more education, which has been strongly positive for everyone.
When I read studies, the intention-to-treat aspect is usually mentioned, and compliance statistics are usually given, but it’s usually communicated in a way that lays traps for people who aren’t reading carefully. Ie, if someone is trying to predict whether the treatment will work for their own three year old, and accurately predicts similar compliance issues, they’re likely to arrive at an efficacy estimate which double-discounts due to noncompliance. And similarly when studies have surprisingly-low compliance, people who expect themselves to comply fully will tend to get an unduly pessimistic estimate of what will happen.
I don’t think D4 works, because the type of cognition it uses (fast-reflex execution of simple patterns provided by a coach) are not the kind that would be affected.
For a long time I’ve observed a pattern that, when news articles talk about Elon Musk, they’re dishonest (about what he’s said, done, and believes), and that his actual writing and beliefs are consistently more reasonable than the hit pieces portray.
Some recent events seem to me to have broken that pattern, with him saying things that are straightforwardly false (rather than complicated and ambiguously-false), and then digging in. It also appeared to me, at the public appearance where he had a chainsaw, that his body language was markedly different from his past public appearances.
My overall impression is that there has been a significant change in his cognitive state, and that he is de facto severely cognitively impaired as compared to how he was a few years ago. It could be transition to a different kind of bipolar, as you speculate, or a change in medications or drug use, or something else. I think people close to him should try coaxing him into doing some sort of cognitive test which has a clear point of comparison, to show him the contrast.
The remarkable thing about human genetics is that most of the variants ARE additive.
I think this is likely incorrect, at least where intelligence-affecting SNPs stacked in large numbers are concerned.
To make an analogy to ML, the effect of a brain-affecting gene will be to push a hyperparameter in one direction or the other. If that hyperparameter is (on average) not perfectly tuned, then one of the variants will be an enhancement, since it leads to a hyperparameter-value that is (on average) closer to optimal.
If each hyperparameter is affected by many genes (or, almost-equivalently, if the number of genes greatly exceeds the number of hyperparameters), then intelligence-affecting traits will look additive so long as you only look at pairs, because most pairs you look at will not affect the same hyperparameter, and when they do affect the same hyperparameter the combined effect still won’t be large enough to overshoot the optimum. However, if you stack many gene edits, and this model of genes mapping to hyperparameters is correct, then the most likely outcome is that you move each hyperparameter in the correct direction but overshooting the optimum. Phrased slightly differently: intelligence-affecting genes may be additive on current margins, but not remain additive when you stack edits in this way.
To make another analogy: SNPs affecting height may be fully additive, but if the thing you actually care about is basketball-playing ability, there is an optimum amount of editing after which you should stop, because while people who are 2m tall are much better at basketball than people who are 1.7m tall, people who are 2.6m tall are cripples.
For this reason, even if all the gene-editing biology works out, you will not produce people in the upper end of the range you forecast.
You can probably somewhat improve this situation by varying the number of edits you do. Ie, you have some babies in which you edit a randomly selected 10% of known intelligence-affecting SNPs, some in which you’ve edited 20%, some 30%, and so on. But finding the real optimum will probably require understanding what the SNPs actually do, in terms of a model of brain biology, and understanding brain biology well enough to make judgment calls about that.
Arbital has been imported to LessWrong
Downvotes don’t (necessarily) mean you broke the rules, per se, just that people think the post is low quality. I skimmed this, and it seemed like… a mix of edgy dark politics with poetic obscurantism?
Any of the many nonprofits, academic research groups, or alignment teams within AI labs. You don’t have to bet on a specific research group to decide that it’s worth betting on the ecosystem as a whole.
There’s also a sizeable contingent that thinks none of the current work is promising, and that therefore buying a little time is value mainly insofar as it opens the possibility of buying a lot of time. Under this perspective, that still bottoms out in technical research progress eventually, even if, in the most pessimistic case, that progress has to route through future researchers who are cognitively enhanced.
The article seems to assume that the primary motivation for wanting to slow down AI is to buy time for institutional progress. Which seems incorrect as an interpretation of the motivation. Most people that I hear talk about buying time are talking about buying time for technical progress in alignment. Technical progress, unlike institution-building, tends to be cumulative at all timescales, which makes it much more strategically relevant.
All of the plans I know of for aligning superintelligence are timeline-sensitive, either because they involve research strategies that haven’t paid off yet, or because they involve using non-superintelligent AI to help with alignment of subsequent AIs. Acceleration specifically in the supply of compute makes all those plans harder. If you buy the argument that misaligned superintelligence is a risk at all, Stargate is a bad thing.
The one silver lining is that this is all legible. The current administration’s stance seems to be that we should build AI quickly in order to outrace China; the previous administration’s stance was to say that the real existential risk is minorities being denied on loan applications. I prefer the “race with China” position because at least there exists a set of factual beliefs that would make that correct, implying it may be possible to course-correct when additional information becomes available.
[The LW crosspost was for some reason pointed at a post on the EA Forum which is a draft, which meant it wouldn’t load. I’m not sure how that happened. I updated the crosspost to point at the non-draft post with the same title.]