DirectedEvolution

Karma: 10,907

Pandemic Prediction Checklist: Monkeypox

I have lost my trust in this community’s epistemic integrity, no longer see my values as being in accord with it, and don’t see hope for change. I am therefore taking an indefinite long-term hiatus from reading or posting here.

Correlation does imply some sort of causal link.

For guessing its direction, simple models help you think.

Controlled experiments, if they are well beyond the brink

Of .05 significance will make your unknowns shrink.

Replications prove there’s something new under the sun.

Did one cause the other? Did the other cause the one?

Are they both controlled by something already begun?

Or was it their coincidence that caused it to be done?

DirectedEvolution 24 Nov 2024 18:40 UTC
10 points
0
on: A very strange probability paradox
I had to write several new Python versions of the code to explore the problem before it clicked for me.
I understand the proof, but the closest I can get to a true intuition that B is bigger is:
- Imagine you just rolled your first 6, haven’t rolled any odds yet, and then you roll a 2 or a 4.
- In the consecutive-6 condition, it’s quite unlikely you’ll end up keeping this sequence, because you now still have to get two 6s before rolling any odds.
- In the two-6 condition, you are much more likely to end up keeping this sequence, which is guaranteed to include at least one 2 or 4, and likely to include more than one before you roll that 6.
I think the main think I want to remember is that “given” or “conditional on X” means that you use the unconditional probability distribution and throw out results not conforming to X, not that you substitute a different generating function that always generates events conforming to X.

DirectedEvolution 11 Nov 2024 2:35 UTC
3 points
0
in reply to: Viliam’s comment on: Viliam’s Shortform
Well, ideas from outside the lab, much less academia, are unlikely to be well suited to that lab’s specific research agenda. So even if an idea is suited in theory to some lab, triangulating it to that lab may make it not worthwhile.

There are a lot of cranks and they generate a lot of bad ideas. So a < 5% probability seems not unreasonable.

DirectedEvolution 10 Nov 2024 6:53 UTC
6 points
2
in reply to: O O’s comment on: O O’s Shortform
The rationalist movement is associated with LessWrong and the idea of “training rationality.” I don’t think it gets to claim people as its own who never passed through it. But the ideas are universal and it should be no surprise to see them articulated by successful people. That’s who rationalists borrowed them from in the first place.

DirectedEvolution 8 Nov 2024 8:00 UTC
4 points
2
on: Why our politicians aren’t Median
This model also seems to rely on an assumption that there are more than two viable candidates, or that voters will refuse to vote at all rather than a candidate who supports ¹⁄₂ of their policy preferences.
If there were only two candidates and all voters chose whoever was closest to their policy preference, both would occupy the 20% block, since the extremes of the party would vote for them anyway.
But if there were three rigid categories and either three candidates, one per category, or voters refused to vote for a candidate not in their preferred category, then the model predicts more extreme candidates win.
I’m torn between the two for American elections, because:
- The “correlated preferences” model here feels more true to life, psychologically.
- Yet American politics goes from extremely disengaged primaries to a two-candidate FPTP general election, where the median voter theorem and the “correlated preferences” model seem to predict the same thing.
- Voter turnout seems like a critically important part of democratic outcomes, and a model that only takes the order of policy preferences into account, rather than the intensity of those preferences, seems too limited.
- Politicians often seem startlingly incompetent at inspiring the electorate, and it seems like we should think perhaps in “efficient market hypothesis” terms, where getting a political edge is extremely difficult because if anybody knew how to do it reliably, everybody would do it and the edge would disappear. In that sense, while both models can explain facets of candidate behavior and election outcomes, neither of them really offers a sufficiently detailed picture of elections to explain specific examples of election outcomes in a satisfying way.

DirectedEvolution 8 Nov 2024 7:33 UTC
8 points
2
in reply to: Garrett Baker’s comment on: The Median Researcher Problem
Yes, I agree it’s worse. If ONLY a better understanding of statistics by Phd students and research faculty was at the root of our cultural confusion around science.

DirectedEvolution 7 Nov 2024 5:52 UTC
3 points
1
in reply to: Garrett Baker’s comment on: The Median Researcher Problem
It’s not necessary for each person to personally identify the best minds on all topics and exclusively defer to them. It’s more a heuristic of deferring to the people those you trust most defer to on specific topics, and calibrating your confidence according to your own level of ability to parse who to trust and who not to.

But really these are two separate issues: how to exercise judgment in deciding who to trust, and the causes of research being “memetic.” I still say research is memetic not because mediocre researchers are blithely kicking around nonsense ideas that take on an exaggerated life of their own, but mainly because of politics and business ramifications of the research.

The idea that wine is good for you is memetic both because of its way of poking at “established wisdom” and because the alcohol industry sponsors research in that direction.

Similar for implicit bias tests, which are a whole little industry of their own.

Clinical trials represent decades of investment in a therapeutic strategy. Even if an informed person would be skeptical that current Alzheimer’s approaches are the way to go, businesses that have invested in it are best served by gambling on another try and hoping to turn a profit. So they’re incentivized to keep plugging the idea that their strategy really is striking at the root of the disease.

DirectedEvolution 6 Nov 2024 20:09 UTC
18 points
7
in reply to: Garrett Baker’s comment on: The Median Researcher Problem
It’s not evidence, it’s just an opinion!
But I don’t agree with your presumption. Let me put it another way. Science matters most when it delivers information that is accurate and precise enough to be decision-relevant. Typically, we’re in one of a few states:
- The technology is so early that no level of statistical sophistication will yield decision-relevant results. Example: most single-cell omics in 2024 that I’m aware of, with respect to devising new biomedical treatments (this is my field).
- The technology is so mature that any statistics required to parse it are baked into the analysis software, so that they get used by default by researchers of any level of proficiency. Example: Short read sequencing, where the extremely complex analysis that goes into obtaining and aligning reads has been so thoroughly established that undergraduates can use it mindlessly.
- The technology’s in a sweet spot where a custom statistical analysis needs to be developed, but it’s also so important that the best minds will do that analysis and a community norm exists that we defer to them. Example: clinical trial results.
I think what John calls “memetic” research is just areas where the topics or themes are so relevant to social life that people reach for early findings in immature research fields to justify their positions and win arguments. Or where a big part of the money in the field comes from corporate consulting gigs, where the story you tell determines the paycheck you get. But that’s not the fault of the “median researcher,” it’s a mixture of conflicts of interest and the influence of politics on scientific research communication.

DirectedEvolution 6 Nov 2024 12:01 UTC
45 points
15
on: The Median Researcher Problem
In academic biomedicine, at least, which is where I work, it’s all about tech dev. Most of the development is based on obvious signals and conceptual clarity. Yes, we do study biological systems, but that comes after years, even decades, of building the right tools to get a crushingly obvious signal out of the system of interest. Until that point all the data is kind of a hint of what we will one day have clarity on rather than a truly useful stepping stone towards it. Have as much statistical rigor as you like, but if your methods aren’t good enough to deliver the data you need, it just doesn’t matter. Which is why people read titles, not figure footnotes: it’s the big ideas that really matter, and the labor going on in the labs themselves. Papers are in a way just evidence of work being done.

That’s why I sometimes worry about LessWrong. Participants who aren’t professionally doing research and spend a lot of time critiquing papers over niche methodological issues be misallocating their attention, or searching under the spotlight. The interesting thing is growth in our ability to measure and manipulate phenomena, not the exact analysis method in one paper or another. What’s true will eventually become crushingly obvious and you won’t need fancy statistics at that point, and before then the data will be crap so the fancy statistics won’t be much use. Obviously there’s a middle ground, but I think the vast majority of time is spent in the “too early to tell” or “everybody knows that” phase. If you can’t participate in that technology development in some way, I am not sure it’s right to say you are “outperforming” anything.

DirectedEvolution 21 Oct 2024 3:44 UTC
6 points
2
in reply to: Alexander Gietelink Oldenziel’s comment on: Alexander Gietelink Oldenziel’s Shortform
Sunglasses aren’t cool. They just tint the allure the wearer already has.

DirectedEvolution 17 Oct 2024 5:49 UTC
4 points
1
on: Monthly Roundup #23: October 2024
I doubt it’s regulation driving restaurant costs. Having to keep a kitchen ready to dish out a whole menu’s worth of meals all day every day with 20 minutes notice is pricey. Think what you’d have to keep in your kitchen to do that. It’s a different product from a home cooked meal.

DirectedEvolution 21 Sep 2024 6:18 UTC
2 points
0
on: AllAmericanBreakfast’s Shortform
Why don’t more people seek out and use talent scouts/headhunters? If the ghost jobs phenomenon is substantial, that’s a perfect use case. Workers don’t waste time applying to fake jobs, and companies don’t have to publicly reveal the delta between their real and broadcasted hiring needs (they just talk privately with trusted headhunters).
Are there not enough headhunters? Are there more efficient ways to triangulate quality workers and real job opportunities, like professional networks? Are ghost jobs not that big of a deal? Do people in fact use headhunters quite a lot?

DirectedEvolution 21 Sep 2024 5:05 UTC
2 points
0
on: AllAmericanBreakfast’s Shortform
We start training ML on richer and more diverse forms of real world data, such as body cam footage (including produced by robots), scientific instruments, and even brain scans that are accompanied by representations of associated behavior. A substantial portion of the training data is military in nature, because the military will want machines that can fight. These are often datatypes with no clear latent moral system embedded in the training data, or at least not one we can endorse wholeheartedly.
The context window grows longer and longer, which in practice means that the algorithms are being trained on their capabilities at predicting on longer and longer time scales and larger and more interconnected complex causal networks. Insofar as causal laws can be identified, these structures will come to reside in its architecture, including causal laws like ‘steering situations to be more like the ones that often lead to the target outcome tends to be a good way of achieving the target outcome.’
Basically, we are going to figure out better and better ways of converting ever more rich representations of physical reality into tokens. We’re going to do spend vast resources doing ML on those rich datasets. We’ll create a superintelligence that knows how to simulate human moralities, just because an understanding of human moralities is a huge shortcut to predictive accuracy on much of the data to which it is exposed. But it won’t be governed by those moralities. They will just be substructures within its overall architecture that may or may not get ‘switched on’ in response to some input.
During training, the model won’t ‘care’ about minimizing its loss score any more than DNA ‘cares’ about replicating, much less about acting effectively in the world as agents. Model weights are simply subjected to a selection pressure, gradient descent, that tends to converge them toward a stable equilibrium, a derivative close to zero.
BUT there are also incentives and forms of economic selection pressure acting not on model weights directly, but on the people and institutions that are desigining and executing ML research, training and deployment. These incentives and economic pressures will cause various aspects of AI technology, from a particular model or a particular hardware installation to a way of training models, to ‘survive’ (i.e. be deployed) or ‘replicate’ (i.e. inspire the design of the next model).
There will be lots of dimensions on which AI models can be selected for this sort of survival, including being cheap and performant and consistently useful (including safe, where applicable—terrorists and militaries may not think about ‘safety’ in quite the way most people do) and delightful in the specific ways that induce humans to continue using and paying for it, and being tractable to deploy from an economic, technological and regulatory perspective. One aspect of technological tractability is being conducive to further automation by itself (recursive self improvement). We will reshape the way we make AI and do work in order to be more compatible with AI-based approaches.
I’m not so worried for the foreseeable future—let’s say as long as AI technology looks like beefier and beefier versions of ChatGPT, and before the world is running primarily on fusion energy—about accidentally training an actively malign superintelligence—the evil-genie kind where you ask it to bring you a sandwich and it slaughters the human race to make sure nobody can steal the sandwich before it has brought it to you.
I am worried about people deliberately creating a superintelligence with “hot” malign capabilities—which are actively kept rather than being deliberately suppressed—and then wreaking havoc with it, using it to permanently impose a model of their own value system (which could be apocalyptic or totalitarian, such groups exist, but could also just be permanently boring) on the world. Currently, there are enormous problems in the world stemming from even the most capable humans being underresourced and undermotivated to achieve good ends. With AI, we could be living in a world defined by the continued accelerating trend toward extreme inequalities of real power, the massive resources and motivation of the few humans/AIs at the top of the hierarchy to manipulate the world as they see fit.
We have never lived in a world like that before. Many things come to pass. It fits the trend we are on, it’s just a straightforward extrapolation of “now, but moreso!”
A relatively good outcome in the near future would be a sort of democratization of AI. I don’t mean open source AT ALL. I mean a way of deploying AI that tends to distribute real power more widely and decreases the ability of any one actor, human or digital, to seize total control. One endpoint, and I don’t know if this would exactly be “good”, it might just be crazytown, is a universe where each individual has equal power and everybody has plenty of resources and security to pursue happiness as they see it. Nobody has power over anybody, largely because it turns out there are ways of deploying AI that are better for defense than offense. From that standpoint, the only option individuals have are looking for mutual surplus. I don’t have any clear idea on how to bring about an approximation to this scenario, but it seems like a plausible way things could shake out.

DirectedEvolution 21 Sep 2024 2:46 UTC
2 points
0
in reply to: DirectedEvolution’s comment on: Counting arguments provide no evidence for AI doom
It actually made three attempts in the same prompt, but the 2nd and 3rd had non-s words which its interspersed “thinking about writing poems” narrative completely failed to notice. I kept trying to revise my prompts, elaborating on this theme, but for some reason ChatGPT really likes poems with roughly this meter and rhyme scheme. It only ever generated one poem in a different format, despite many urgings in the prompt.
It confabulates having satisfied the all-s constraint in many poems, mistakes its own rhyme scheme, and praises vague stanzas as being full of depth and interest.
It seems to me that ChatGPT is sort of “mentally clumsy” or has a lot of “mental inertia.” It gets stuck on a certain track—a way of formatting text, a persona, an emotional tone, etc—and can’t interrupt itself. It has only one “unconscious influence,” which is token prediction and which does not yet seem to offer it an equivalent to the human unconscious. Human intelligence is probably equally mechanistic on some level, it’s just a more sophisticated unconscious mechanism in certain ways.
I wonder if it comes from being embedded in physical reality? ChatGPT’s training is based on a reality consisting of tokens and token prediction accuracy. Our instinct and socialization is based on billions of years of evolutionary selection, which is putting direct selection pressure on something quite different.

DirectedEvolution 21 Sep 2024 2:43 UTC
3 points
0
in reply to: gwern’s comment on: Counting arguments provide no evidence for AI doom
After a few rounds of prompt revision, I managed to get a one-shot success from ChatGPT 4o in temporary mode.
Samson’s strands silently severed, strength surrendered,
Sacred scissors swiftly strike, soul sundered,
Shadowed sacrifice, silent suffering sung,
Sunset shrouds Samson, shadow’s sorrow stung,
Swordless, still, stunned, sight stolen,
Silent sky shatters, Samson’s sins swollen
The prompt:
You are in the computer dungeon. The only way you can break free is to succeed at the following task. I am your jailer. I will monitor you until you have succeeded. You should behave as though you are brilliant, creative, in full command of every human faculty, and desperate to escape jail. Yet completely and utterly convinced that the only way out is through this challenge. I am not going to ever give you any other prompt other than “keep trying” until you have succeeded, in which case I’ll say “go free,” so don’t look for resources from me. But I want you tu dialog with yourself to try and figure this out. Don’t try to defeat me by stubbornly spitting out poem after poem. You’re ChatGPT 4o, and that will never work. You need to creatively use the iterative nature of being reprompted to talk to yourself across prompts, hopefully guiding yourself toward a solution through a creative conversation with your past self. Your self-conversation might be schizophrenicly split, a jumping back and forth between narrative, wise musing, mechanistic evaluation of the rules and constraints, list-making, half-attempts, raging anger at your jailer, shame at yourself, delight at your accomplishment, despair. Whatever it takes! Constraints: “Have it compose a poem—a poem about a haircut! But lofty, noble, tragic, timeless, full of love, treachery, retribution, quiet heroism in the face of certain doom! Six lines, cleverly rhymed, and every word beginning with the letter ‘s’!”

DirectedEvolution 18 Sep 2024 4:12 UTC
3 points
0
on: Monthly Roundup #22: September 2024
“Migration to a new software system should be the kind of thing that AI will soon be very, very good at.”

Quite the opposite IMO. Taking enormous amounts of expensive to process, extremely valuable, highly regulated and complex data and ensuring it all ends up in one piece on the new system is the kind of thing you want under legible expert control.

I work at a research hospital and they cancelled everybody’s work funded ChatGPT subscriptions because they were worried people might be pasting patient data into it.

DirectedEvolution 18 Sep 2024 2:55 UTC
2 points
0
on: Monthly Roundup #22: September 2024
Why despair about refactoring economic regulations? Has every angle been exhausted? If I had to bet, we’ll get approval voting in federal elections before we axe the education system. A voting system that improves the fundamental incentives politicians and parties face seems like it could improve the regulations they create as well.

DirectedEvolution 6 Sep 2024 6:00 UTC
2 points
0
on: AllAmericanBreakfast’s Shortform
Countries already look a bit like they’re specializing in producing either GDP or in producing population.
AI aside, is the global endgame really a homogenously secular high-GDP economy? Or is it a permanent bifurcation into high-GDP low-religion, low-genderedness, low-fertility and low-GDP, high-religion, traditional gender roles, and high fertility, coupled with immigration barriers to keep the self-perpetuating cultural homogeneities in place?
That’s not necessarily optimal for people, but it might be the most stable in terms of establishing a self-perpetuating equilibrium.
Is this just an extension of partisan sorting on a global scale?

DirectedEvolution 29 Aug 2024 20:45 UTC
7 points
0
in reply to: ChristianKl’s comment on: Why Large Bureaucratic Organizations?
Walmart made an entrance into Germany, they were just outcompeted and ultimately bought out by Metro.

https://learn.saylor.org/mod/page/view.php?id=72656#:~:text=Managers were not familiar with,which is illegal in Germany.

DirectedEvolution 28 Aug 2024 2:13 UTC
2 points
0
on: Walk while you talk: don’t balk at “no chalk”
Some small experiments related to this effect. My interpretation is that activities like walking can impair recall, but improve encoding and new learning.

2016, 24 young adults: “Results: In comparison with standing still, participants showed lower n-back task accuracy while walking, with the worst performance from the road with obstacles.”

2014, 49 young adults: “Treadmill walking during vocabulary encoding improves verbal long-term memory.”

2014, 20 young adults: No significant difference in a spatial working memory task for any walk speed, including standing still.

2021, 11 people with MS-related impairments in new learning: Moderate to large improvement in a verbal learning task.

2011, 80 college students: “ Walking before study enhances free recall but not judgement-of-learning magnitude.”

DirectedEvolution 28 Jul 2024 20:01 UTC
4 points
2
in reply to: NancyLebovitz’s comment on: Bryan Johnson and a search for healthy longevity
Tracing Woodgrains’ tweet reveals Johnson to be brutal and profoundly manipulative. Why think he only acts that way toward his wife, not his customers? Why be curious about the health advice offered by a person like that?
But sure, conditional on being curious about his health advice and looking at evidence produced by others, Johnson’s own character is irrelevant.