I am very confused.
My first thought when reading this was ‘huh, no wonder they’re getting mixed results—they’re doing it wrong’.
My second thought when returning to this a day later: good—anything I do to contribute to the ability to understand and measure persuasion is literally directly contributing to dangerous capabilities.
Counterfactually, if we don’t create evals for this… are we not expected to notice that LLMs are becoming increasingly more persuasive? More able to model and predict human psychology?
What is actually the ‘safety’ case for this research? What theory of change predicts this work will be net positive?
stavros
Karma: 219
Care Doesn’t Scale
stavros’s Shortform
Re: 2
Most promising way is just raising children better.
See (which I’m sure you’ve already read): https://www.lesswrong.com/posts/CYN7swrefEss4e3Qe/childhoods-of-exceptional-people
Alongside that though, I think the next biggest leverage point would be something like nationalising social media and retargeting development/design toward connection and flourishing (as opposed to engagement and profit).
This is one area where, if we didn’t have multiple catastrophic time pressures, I’d be pretty optimistic about the future. These are incredibly high impact and tractable levers for changing the world for the better; part of the whole bucket of ‘just stop doing the most stupid thing’ stuff.
Is there anything useful we can learn from Crypto ASICs as to how this will play out? And specifically, how to actually bet on it?
Replying to this because it seems a useful addition to the thread; assuming OP already knows this (and more).
1.) And none of the correct counterplays are ‘look, my opponent is cheating/look, this game is unfair’. (Scrub mindset)
2.) You know what’s more impressive than winning a fair fight? Winning an unfair one. While not always an option, and usually with high risk:reward, beating an opponent who has an assymetric situational advantage is hella convincing; it affords a much higher ceiling (relative to a ‘fair’ game) to demonstrate just how much better than your opponent you are.
It’s an interesting framework, I can see it being useful.
I think it’s more useful when you consider both high-decoupling and low-decoupling to be failure modes, more specifically: when one is dominant and the other is neglected, you reliably end up with inacccurate beliefs.You went over the mistakes of low-decouplers in your post, and provided a wonderful example of a high-decoupler mistake too!
High decouplers will notice that, holding preferences constant, offering people an additional choice cannot make them worse off. People will only take the choice if its better than any of their current options
Aside from https://thezvi.wordpress.com/2017/08/12/choices-are-really-bad/ there’s also the consideration of what choice I offer you, or how I frame the choice (see Kahneman’s stuff).
And that’s just considering it from the individual psychological level, but there are social/cultural levers and threads to pull here too.
I think the optimal functioning of this process is cyclical with both high decoupling phases and highly integrated phases, and the measure of balance is something like ‘this isn’t obviously wrong in either context’.
I think future technology all has AI as a pre-requisite?
My high conviction hot take goes further: I think all positive future timelines have AI as a pre-requisite. I expect that, sans AI, our future—our immediate future: decades, not centuries—is going to be the ugliest, and last, chapter in our civilization’s history.
I have been in the position of trying to moderate a large and growing community—it was at 500k users last I checked, although I threw in the towel around 300k—and I know what a thankless, sisyphean task it is.
I know what it is to have to explain the same—perfectly reasonable—rule/norm again and again and again.
I know what it is to try to cultivate and nurture a garden while hordes of barbarians trample all over the place.
But...
If it aint broke, don’t fix it.
I would argue that the majority of the listed people penalized are net contributors to lesswrong, including some who are strongly net positive.
I’ve noticed y’all have been tinkering in this space for a while, I think you’re trying super hard to protect lesswrong from the eternal september and you actually seem to be succeeding, which is no small feat, buuut...
I do wonder if the team needs a break.
I think there’s a thing that happens to gardeners (and here I’m using that as a very broad archetype), where we become attached to and identify with the work of weeding—of maintaining, of day after day holding back entropy—and cease to take pleasure in the garden itself.
As that sets in, even new growth begins to seem like a weed.
Fine. You win. Take your upvote.
Big fan of both of your writings, this dialogue was a real treat for me.
I’ve been trying to find a satisfying answer to the seeming inverse correlation of ‘wellbeing’ and ‘agency’ (these are very loose labels).
You briefly allude to a potential mechanism for this[1]
You also briefly allude to another mechanism with explanatory power for the inverse[2] - i.e. that while it might seem an individual is highly agentic, they are in fact little more than a host for a highly agentic egregore
I’m engaged in that most quixotic endeavour of actually trying to save the world[3] [4], and thus I’m constantly playing with my world model and looking for levers to pull, dominos to push over, that might plausibly -and quickly- shift probability mass towards pleasant timelines.
I think germ theory is exactly the kind of intervention that works here—it’s a simple map that even a child can understand, yet it’s a 100x impact.
I think there’s some kind of ‘germ theory for minds’, and I think we already have all the pieces—we just need to put them together in the right way. I think it’s plausible that this is easy, rapidly scaleable and instrumentally valuable to other efforts in the ‘save the world’ space.
But… I don’t want to end up net negative on agency. In fact my primary objective is to end up strongly net positive. I need more people trying to change the world, not less.
Yet… that scale of ambition seems largely the preserve of people you’d be highly unlikey to describe as ‘enlightened’, ‘balanced’ or ‘well adjusted’; it seems to require a certain amount of delusion to even (want to) try, and benefit from unbalanced schema that are willing to sacrifice everything on the altar of success.Most of the people who seem to succcessfully change the world are the people I least want to; whereas the people I most want to change the world seem the least likely to.
- ^
Since the schools that removed social conditioning and also empowered practitioners to upend the social order, tended to get targeted for destruction. (Or at least so I suspect and some people on Twitter said “yes this did happen” when I speculated this out loud.)
- ^
In the Buddhist model of human psychology, we are by default colonized by parasitic thought patterns, though I guess in some cases, like the aforementioned fertility increasing religious memes, they should be thought of as symbiotes with a tradeoff, such as degrading the hosts’ episteme.
- ^
I don’t expect to succeed, I don’t expect to even matter, but it’s a fun hobby.
- ^
Also the world does actually seem to be in rather urgent need of saving; short of a miracle or two it seems like I’m unlikely to live to enjoy my midlife crisis.
- ^
I don’t think there’s anything wrong with cultivating a warrior archetype; I strive to cultivate one myself.
Would love to read more on this.
+1 for the 14⁄24 club.
Hmmm, where to start. Something of a mishmash of thought here.
Actually a manager, not yet clear if I’m particularly successful at it. I certainly enjoy it and I’ve learned a lot in the past year.
Noticing Panic is a great Step 0, and I really like how you contrast it to noticing confusion.
I used to experience ‘Analysis Paralysis’ - too much planning, overthinking, and zero doing. This is a form of perfectionism, and is usually rooted in fear of failure.
I expect most academics have been taught entirely the wrong (in the sense of https://www.lesswrong.com/posts/a7n8GdKiAZRX86T5A/making-beliefs-pay-rent-in-anticipated-experiences) heuristics around failure.
My life rapidly became more agentic the more I updated toward the following beliefs:
Failure is cheap
You have an abundance of chances to get it right
Plans are maps, reality is terrain. Doing happens in reality. Thus you can offload a bunch of cognitive work to reality simply by trying stuff; failing is one of the most efficient ways of updating your map, and can sometimes reward you with unexpected success (take a look around at all the ‘stupid’ stuff that actually works/succeeded).
Thus my management strategy is something like:
Have a goal
Build a model of the factors that contribute to that goal
Determine my constraints (e.g. do I have a rigid deadline)
Notice my affordances (most people always underestimate this)
What resources do I have, e.g.
Who can I ask for help?
What work has already been done that I can use (don’t reinvent the wheel)
What actions are available to me
What is the smallest meaningful step I can take toward my goal?
What is the dumbest thing I can do that might actually work?
Prioritize my time
What needs to be done today vs this week vs this month vs actually doesn’t need to be done
So I’ve reduced a combinatorially explosive long term goal into a decent heuristic for prioritizing actions, and then I apply it to the actions I can actually take at different timescales… which is usually an easy choice between a handful of options.
Then I do stuff, and then I update/iterate based on the results.
And sometimes stuff just happens that moves me towards my goal (or my goal towards me) - life is chaotic, and if you’re rigidly following a plan then that chaos is always working against you. Whereas if you’re adaptable and opportunistic—that chaos can work for you.
I guess all of this boils down to: invest in your world model, not your plan.
Re: average age of authors/laureates and average team size
Are these data adjusted for demographic changes? i.e. Aging populations in most western countries, and general population growth.
I think this is a mistake to import “democracy” at the vision level. Vision is essentially a very high-level plan, a creative engineering task. These are not decided by averaging opinions. “If you want to kill any idea in the world, get a committee working on it.” Also, Deutsch was writing about this in “The Beginning of Infinity” in the chapter about democracy.
We should aggregate desiderata and preferences (see “Preference Aggregation as Bayesian Inference”), but not decisions (plans, engineering designs, visions). These should be created by a coherent creative entity. The same idea is evident in the design of Open Agency Architecture.
Democracy is a mistake, for all of the obvious reasons.
As is the belief amongst engineers that every problem is an engineering problem :PWe have a whole bunch of tools going mostly unused and unnoticed that could, plausibly, enable a great deal more trust and collaboration than is currently possible.
We have a whole bunch of people both thinking about and working on the polycrisis already.My proposal is that we’re far more likely to achieve our ultimate goal—a future we’d like to live in—if we simply do our best to empower, rather than direct, others.
I expect attempts to direct, no matter how brilliant the plan or the mind(s) behind it, are likely to fail. For all the obvious reasons.
(caveat: yes AGI changes this, but it changes everything. My whole point is that we need to keep the ship from sinking long enough for AGI to take the wheel)
Joshua Williams created an initial version of a metacrisis map
It’s a good presentation, but it isn’t a map.
A literal map of the polycrisis[1] can show:
The various key facets (pollution, climate, biorisk, energy, ecology, resource constraints, globalization, economy, demography etc etc)
Relative degrees of fragility / timelines (e.g. climate change being one of the areas where we have the most slack)
Many of the significant orgs/projects working on these facets, with special emphasis placed on those that are aware of the wider polycrisis
Many of the significant communities
Many of the significant funders
Do you mean that it’s possible to earn by betting long against the current market sentiment?
- ^
I mildly prefer polycrisis because it’s less abstract. The metacrisis points toward a systems dynamic for which we have no adequate levers, whereas the polycrisis points toward the effects in the real world that we need to deal with.
I am assuming we live in a world that is going to be reshaped (or ended) by technology (probably AGI) within a few decades, and that if this fails to occur the inevitable result of the metacrisis is collapse.
I think the most impact I can have is to kick the can down the road far enough that the accelerationistas get their shot. I don’t pretend this is the world I would choose to be living in, or the horse I’d want to be betting on. It is simply my current understanding of reality.Hence: polycrisis. Deal with the symptoms. Keep the patient alive.
The polycrisis has been my primary source of novelty/intellectual stimulation for a good long while now. Excited to see people explicitly talking about it here.
With regard to the central proposition:
I think if there were A Plan to make the world visibly less broken, made out of many components which are themselves made out of components that people could join and take responsibility for, this would increase the amount of world-fixing work being done and would meaningfully decrease the brokenness of the world. Further, I think there’s a lot of Common Cause of Many Causes stuff going on here, where people active in this project are likely to passively or actively support other parts of this project / there could be an active consulting / experience transfer / etc. scene built around it.
I think this is largely sensible and true, but consider top-down implementation of such to be a pipe dream.
Instead there is a kind of grassroots version where you do some combination of:
1.) Clearly state the problems that need to be worked on, and provide reasonable guidance as to where and how they might be worked on
2.) Notice what work is already being done on the problems, and who is doing it (avoid reinventing the wheel/not invented here syndrome; EA is especially guilty of this)
3.) Actively develop useful connections between 2.)
4.) Measure engagement (resource flows) and progress
And from that process I expect something like a plan to emerge—it won’t be the best possible plan, but it will be far from the worst plan, more adequate than not, and importantly it will survive contact with reality because reality was a key driver in the development of the plan.The platform for generating the plan would need to be more-open-than-not, and should be fairly bleeding edge—incorporating prediction markets, consensus seeking (polis), eigenkarma etc
It should be a design goal that high value contributions should be noticed, no matter the source. An example of this actually happening is where Taiwan was able to respond rapidly to Covid thanks to a moderator noticing and doing due diligence on a post in the .g0v forums re: covid, and having a process in place where that information could be escalated to government.
It should also be subject to a serious amount of adversarial testing—such a platform, if successful, will influence $ flows, and thus will be a target for capture/gaming etc etc.As it stands, we’re lacking all 4. We’re lacking a coherent map of the polycrisis[1], we’re lacking in useful+discoverable communication channels, we’re lacking meaningful 3rd party measurement.
As it stands, the barriers to entry for those wishing to engage in meaningful work in this space are absurd.
If you lack the credentials and/or wealth to self-fund, then you’re effectively excluded—a problem which was created by an increasingly specialized world (And the worldview, cultural dynamics and behaviours it engenders) has gatekeepers from that same world, enforcing the same bottlenecks/selective pressures of that world on those who would try to solve the problem.The neighbourhood is on fire, and the only people allowed to join the bucket chain are those most likely to be ignoring the fire—so very catch-22.
P.S.
I think there’s a ton of funding available in this space, specifically I think speculating on the markets informed by the kind of worldview that allows one to perceive the polycrisis has significant alpha. I think we can make much better predictions about the next 5-10 years than the market, and I don’t think most of the market is even trying to make good predictions on those timescales.
I’d be interested in talking/collaborating with anyone who either strongly agrees or disagrees with this logic.
- ^
On this note, if anyone wants to do and/or fund a version of aisafety.world for the polycrisis, I’m interested in contributing.
- ^
There’s a guy called Rafe Kelley on youtube who has a fairly good answer to this, which I’m going to attempt to summarize from memory because I can’t point you toward any reasonable sources (I heard him talking about it in a 1h+ conversation with everyone’s favourite boogeyman, Jordan Peterson).
His reasoning goes thus:
1.) We need play in order to develop: play teaches us how to navigate Agent—Arena relationships
This speaks to the result of playground injuries increasing despite increased supervision—kids aren’t actually getting to spend enough time playing in the physical Arena, their capability to navigate it is underdeveloped because of excess indoor time and excess supervision.
2.) We need rough play (e.g. play fighting), specifically, to teach us a whole bunch of capabilities around Agent—Agent—Arena relationships; conflict, boundaries, emotional regulation are all, Rafe argues, rooted in rough play.
Through rough and tumble play, we learn the physical boundaries between agents. We learn that it hurts them, or us, when boundaries are crossed. We learn where those boundaries are. We learn to regulate our emotions with respect to those boundaries.
These are highly transferable, core skills, without which human development is significantly stunted.
See Also: Catch-22