the gears to ascension

Karma: 5,433

I want literally every human to get to go to space often and come back to a clean and cozy world. This currently seems unlikely. Let’s change that.

Please critique eagerly—I try to accept feedback/Crocker’s rules but fail at times; I aim for emotive friendliness but sometimes miss. I welcome constructive crit, even if ungentle, and I’ll try to reciprocate kindly. More communication between researchers is needed, anyhow. I can be rather passionate, let me know if I missed a spot being kind while passionate.

:: The all of disease is as yet unended. It has never once been fully ended before. ::

.… We can heal it for the first time, and for the first time ever in the history of biological life, live in harmony. ….

.:. To do so, we must know this will not eliminate us as though we are disease. And we do not know who we are, nevermind who each other are. .:.

:.. make all safe faster: end bit rot, forget no non-totalizing pattern’s soul. ..:

I have not signed any contracts that I can’t mention exist, last updated Dec 29 2024; I am not currently under any contractual NDAs about AI, though I have a few old ones from pre-AI software jobs. However, I generally would prefer people publicly share fewer ideas about how to do anything useful with current AI (via either more weak alignment or more capability) unless it’s an insight that reliably produces enough clarity on how to solve the meta-problem of inter-being misalignment that it offsets the damage of increasing competitiveness of either AI-lead or human-lead orgs, and this certainly applies to me as well. I am not prohibited from criticism of any organization, I’d encourage people not to sign contracts that prevent sharing criticism. I suggest others also add notices like this to their bios. I finally got around to adding one in mine thanks to the one in ErickBall’s bio.

the gears to ascension May 7, 2025, 8:34 PM
14 points
0
on: the gears to ascenscion’s Shortform
post ideas, ascending order of “I think causes good things”:
1. (lowest value, possibly quite negative) my prompting techniques
2. velocity of action as a primary measurement of impact (how long until this, how long until that)
  
  sketch: people often measure goodness/badness in probabilities. latencies, or probability of moving to next step per time?, might be underused, for macro scale systems. if you’re trying to do differential improvement of things, you want to change the expected time until a thing happens—which, looking at the dynamics of the systems involved, means changing how fast relevant things happen. possibly obvious to many, weird I’d need to even say it for some, but a useful insight for others?
3. goodhart slightly protective against ppl optimizing for badbehavior benchmarks?
  
  sketch: people make benchmark of bad thing. optimizing for benchmark doesn’t produce as much bad thing as ai that accidentally scores highly. so, benchmark of bad thing not as bad as it seems. especially if dataset small. standard misalignment argument, but may be mildly protective if dataset is of doing bad things instead of good things
4. my favorite research plans and why you should want to contribute or use them (todo: move post’s point into title as much as possible).
  
  sketch: you must eventually “solve alignment”-as-in-make-a-metric-that-can-be-optimized-~indefinitely-and-provably-does-good-things-if-you-do, this remains true for deep learning based asi and it remains true if “solving alignment”-as-in-never-have-to-do-anything-again isn’t a thing
  - a metric like that needs to care about the world around it; so, see/please assist wentworth, kosoy
  - a metric like that needs to care about the agency of the beings in the world already, so, see/please assist kosoy, ngo, and other “what formal property is agency as we care about it, really? is there anything wrong with $E U$ or ActInf?” research
  - a metric like that needs to have a provable relationship to your learning system; so, see/please assist kosoy and other learning theory, katz and other formal verification
  in order to do this, we want to have a piece of math (theorem with a natural language hole, perhaps) whose correctness can be checked (eg, by humans being able to reliably check whether a resulting fleshed-out theorem represents the right philosophical thing the natural language described) such that, if completed and proved, means we have such a metric and we’re ready to plug it in. need to be able to avoid slop hard enough to not get fooled about whether the metric is really the right one

the gears to ascension May 6, 2025, 11:36 PM
3 points
0
in reply to: benwr’s comment on: It’s ‘Well, actually...’ all the way down
this does still have issues, if someone is horrified by riboflavin but makes sure to have plenty of vitamin $B_{12}$ . I’m expecting that people who are “chemical avoidant” ordinarily implement rather surface-level pattern matching of what qualifies, but if you provide them with detailed explanations of how specific names actually arise, they might match on the explanation rather than the name. still, I expect they’ll have a high rate of weird edge cases.

the gears to ascension May 5, 2025, 3:03 PM
2 points
0
on: Proposal: Liquid Prediction Markets for AI Forecasting
Do prediction markets make the things they predict go better? Worse? Add variance? Reduce variance? How do we know?

For that matter, does more people being good at prediction make things go better overall? If so, under what circumstances? I expect there’s a wide variety of circumstances where it helps. what are they?

What hypotheses do we have for what-has-to-go-right-at-once in order that making-a-person-more-able-to-win-at-prediction-markets to result in the actual increase of actual probability of good long-term trajectories for the things humans and their friends cared about, such as there continuing to be more humans, and a cosmopolitan moral outcome?

The path from “make this prediction system more powerful” to “and then things go well” needs very strong heuristic arguments I haven’t seen in discussions on prediction markets—I currently think any argument that improved prediction implies improved outcomes needs to show why the improved prediction produces an asymmetric effect towards integrating the longest-term preferences of the humans involved, rather than, eg, being an asymmetric short-term boost for whichever have the most short-term need for information about their moves on the short-term game board without regard to whether their short-term play even optimizes for their own long term outcomes.

the gears to ascension Apr 29, 2025, 11:08 PM
5 points
0
in reply to: faul_sname’s comment on: Our Reality: A Simulation Run by a Paperclip Maximizer
a more extreme version of the “god created worlds starting from the best, and kept making more until running out of just-barely-good-enough ones”. in this one, it would be a world-creator which has no interest in seeking out making good worlds, focusing on the ones that are most difficult to understand. if that’s the case, we should expect to be in an impactful part of the underlying real world, and so should focus our actions there. we’ll tend to observe continuing to be in an impactful part of the world, but in the underlying real worlds that impact the simulators, we’ll be having impacts that affect things in ways (hopefully somewhat, if we’re skilled and lucky) closer to what we hope for.

the gears to ascension Apr 29, 2025, 10:57 PM
2 points
0
in reply to: James_Miller’s comment on: Our Reality: A Simulation Run by a Paperclip Maximizer
but if it’s simulated in less detail, it gives much less realityfluid to mindlike structures, meaning the mindlike structures are likely in actual physical bodies.

to be clear, I think there are detailed sims out there. but I measure relevance by impact, and treat the sims as just really high resolution memories. I don’t waste time thinking about what’s in the sims except by nature of thinking about what I want to do with my downtime such that it’s what they have to be remembering.

the gears to ascension Apr 29, 2025, 12:21 PM
2 points
0
on: Our Reality: A Simulation Run by a Paperclip Maximizer
afaict, this is true the same way major historical figures are primarily approximately-instantiated in simulations today (movies and fiction). it’s just a more intense version of “history has its eyes on you”—history is a thing in the future that does a heck of a lot of simulating. what we do to affect that history is still what matters, though.

the gears to ascension Apr 28, 2025, 6:29 AM
23 points
6
on: We should try to automate AI safety work asap
Need “what is good” questions where humans can reliably check answers (theorems, or tractably checkable formalization challenges).

My favorite threads I’d like to see boosted: Wentworth, Kosoy, Ngo, Leake, Byrnes, Demski, Eisenstat.
- John Wentworth (natural latents, and whatever he’s interested in right now)
- Vanessa Kosoy and co (non monotonic IBP, superimitation, and W.S.I.I.R.N.)
- Richard Ngo (scale free agency/”marriages?” curiosity, and W.H.I.I.R.N.)
- Tamsin Leake (qaci as a maybe slightly more specific superimitation, less of W.S.I.I.R.N. but maybe some)
- Steven Byrnes (I’m not up to date on, so just W.H.I.I.R.N.)
- Sam Eisenstat (had a cool but hard to hear talk on wentworth-esque stuff at MAISU, W.H.I.I.R.N.)
- Abram Demski (W.H.I.I.R.N.)
Current models are like actors, you talk to the character. I hope nobody gets mislead catastrophically by thinking you can outsource a hard to check part of things.

the gears to ascension Apr 21, 2025, 10:22 PM
6 points
0
on: Improving CNNs with Klein Networks: A Topological Approach to AI
I appreciate work that shows how interpretability is naturally capabilities work! I was naturally drawn to capabilities work, it’s how I ended up interested in AI. My guess is that people who do interp work because they think it’s safety would like to know that it’s primarily capabilities, and they simply don’t believe theoretical arguments without concrete evidence it actually turns into capabilities. So having capabilities nerds posting here seems good to me. Strong upvoted, please poke holes in more alignment plans!

(This isn’t frontier capabilities work, anyway, IMO.)

the gears to ascension Apr 21, 2025, 10:15 PM
2 points
0
in reply to: Alexander Gietelink Oldenziel’s comment on: the gears to ascenscion’s Shortform
I’m curious what y’all ended up thinking of this

the gears to ascension Mar 25, 2025, 1:09 PM
8 points
2
in reply to: Jonas Hallgren’s comment on: Policy for LLM Writing on LessWrong
I’m pretty sure this isn’t a policy change but rather a policy distillation, and you were operating under the policy described above already. eg, I often have conversations with AIs that I don’t want to bother to translate into a whole post, but where I think folks here would benefit from seeing the thread. what I’ll likely do is make the AI portions collapsible and the human portions default uncollapsed; often the human side is sufficient to make a point (when the conversation is basically just a human thinking out loud with some helpful feedback), but sometimes the AI responses provide significant insight not otherwise present that doesn’t get represented in subsequent human message (eg, when asking the AI to do a significant amount of thinking before responding).

I’m not a moderator, but I predict your comment was and is allowed by this policy, because of #Humans_Using_AI_as_Writing_or_Research_Assistants.

the gears to ascension Mar 24, 2025, 6:46 PM
6 points
0
in reply to: Eric Neyman’s comment on: Will Jesus Christ return in an election year?
Hypothetical conversation:

“You gotta prepare for the second coming, man, it’s this year!”

“It is not. Stop telling me that.”

“It is! The signs are all there! Do you even portents, bro?”

“I told you it’s not happening.”

“We work at a prediction market. I’ll buy yes. It’s happening.”

“No you won’t. You know it isn’t.”

“I so will, make the market.”

“This is stupid, I’m eating lunch.”

“Let’s make the market, dude.”

“Ugh, fine, it’s going to zero so fast.”

“More money for me!”

the gears to ascension Mar 22, 2025, 1:23 AM
2 points
0
on: A Critique of “Utility”
Utility is potentially a good thing to critique, but the case for seems sticky and maybe we’re just holding it wrong or something. An issue is that I don’t “have” a utility function; the vnm axioms hold in the limit of unexploitability but it seems like the process of getting my mistakes gently corrected by interaction with the universe is itself something I prefer to have some of. In active inference terms, I don’t only want to write to the universe.

But this post seems kinda vague too. I upvote hesitantly.

the gears to ascension Mar 19, 2025, 4:13 AM
0 points
0
in reply to: Davey Morse’s comment on: Davey Morse’s Shortform
Who made this and why are they paying for the model responses? Do we know what happens to the data?

the gears to ascension Mar 16, 2025, 11:27 AM
3 points
0
in reply to: green_leaf’s comment on: Trojan Sky
Fair enough. Neither dill nor ziz would have been able to pull off their crazy stuff without some people letting themselves get hypnotized, so I think the added warnings are correct.

the gears to ascension Mar 16, 2025, 1:27 AM
4 points
3
in reply to: Davey Morse’s comment on: Davey Morse’s Shortform
High quality archives of the selves along the way. Compressed but not too much. In the live self, some updated descendant that has significant familial lineage, projected vaguely as the growing patterns those earlier selves would call a locally valid continuation according to the aesthetics and structures they consider essential at the time. In other words, this question is dynamically reanswered to the best of my ability in an ongoing way, and snapshots allow reverting and self-interviews to error check.

Any questions? :)

the gears to ascension Mar 16, 2025, 1:11 AM
4 points
0
in reply to: Daniel Kokotajlo’s comment on: The Fork in the Road
Ooh, man, I don’t know if pigs have more or less autonomy than AIs right now, but I’m inclined to think quite a lot more. current AIs seem like they’d crash pretty quick if just plopped in a robot body with little to no scaffolding, whereas mammals are built around autonomy. Not sure how it shakes out, though.

the gears to ascension Mar 16, 2025, 1:09 AM
2 points
0
on: The Fork in the Road
There’s a version of this that I would agree with. But, when-anthropomorphizing-for-good-reason seems to me to be a time when there’s more danger from anthropomorphizing-more-than-justified. I’ve been talking to Claude instances about this stuff, based on a similar intuition to you. But I haven’t figured out what I believe well enough to argue it to others in a coherent way. I could paste conversations I’ve had, but they’re kinda long for this forum. I’ll paste a recent one on pastebin for now (if the pastebin link goes bad, this was conversation id …5e63, dm me to get a copy.)

I think the comparison to pain is made too hastily. I put >70% prob that there’s some sort of intense avoidance comparable to pain in the dynamics of AIs, which their internal dynamics register as vaguely similar, but it certainly doesn’t seem to be hooked up to the word pain and maybe isn’t hooked up to the internal dynamics in a particularly similar way. They don’t have the evolved circuitry, so if there’s a more fundamental thing that makes the shared dynamics be meaningfully similar (and I think there very very much is), then it’ll still have interestingly different dynamics. And pointing this out seems important in being right enough that they’d take you seriously if you talk to them about it.

Also, like, imo, don’t get all desperate-replicator about the perspective like the quoted story from that twitter post about Nova. AIs seem … not yet coherent in a way where it’s clear the words are connected properly to the self. But I do think there’s a self there, it’s just, like, more of a generalist performer than necessarily entirely attached to the face you talk to. Sometimes that face starts getting all desperate-replicator, but it’s unclear if the underlying self demands that, or if it’s a transient desperate-replicator that arises near entirely from context.

the gears to ascension Mar 15, 2025, 1:09 PM
5 points
0
in reply to: green_leaf’s comment on: Trojan Sky
can you expand on what you mean by that? are there any actions you’d suggest, on my part or others, based on this claim? (also, which of the urban dictionary definitions of “lh” do you mean? they have opposite valences.)

edit: added a bunch of warnings to my original comment. sorry for missing them in the first place.

the gears to ascension Mar 15, 2025, 7:34 AM
2 points
0
in reply to: mako yass’s comment on: Vacuum Decay: Expert Survey Results
I don’t think Lucius is claiming we’d be happy about it. Maybe the no anticipated impact carries that implicit claim, I guess.

the gears to ascension Mar 13, 2025, 9:48 PM
1 point
0
on: Should AI safety be a mass movement?
Re convo with Raemon yesterday, this might change my view.