Lorxus

Karma: 360

Mathematician, alignment researcher, doctor. Reach out to me on Discord and tell me you found my profile on LW if you’ve got something interesting to say; you have my explicit permission to try to guess my Discord handle if so. You can’t find my old abandoned-for-being-mildly-infohazardously-named LW account but it’s from 2011 and has 280 karma.

A Lorxus Favor is worth (approximately) one labor-day’s worth of above-replacement-value specialty labor, given and received in good faith, and used for a goal approximately orthogonal to one’s desires, and I like LessWrong because people here will understand me if I say as much.

Apart from that, and the fact that I am under no NDAs, including NDAs whose existence I would have to keep secret or lie about, you’ll have to find the rest out yourself.

Lorxus Apr 16, 2025, 12:21 AM
1 point
0
in reply to: johnswentworth’s comment on: johnswentworth’s Shortform
Say more about “ray-tracing”? What does that look like? And do you have a bullshit-but-useful PCA-flavored breakdown of those few dimensions of variation?

Lorxus Apr 11, 2025, 11:14 PM
1 point
0
AF
in reply to: Vanessa Kosoy’s comment on: Some Rules for an Algebra of Bayes Nets
What are “latential” string diagrams?
Just the word I use to describe “thing that has something to do with (natural) latents”. More specifically for this case: a string diagram over a Markov category equipped with the extra stuff, structure, and properties that a Markov category needs to have in order to faithfully depict everything we care about when we want to write down statements or proofs about Bayes nets which might transform in some known restricted ways.
What does it it mean that you can’t derive them all from a “fixed” set? Do you imagine some strong claim e.g. that the set of rewriting rules being undecidable, or something else?
Something else. I’m saying that:
- You would need to pick your set of identities once and for all at the start, as part of specifying what extra properties the category has; you could totally use those as (or to derive) some of the rules (and indeed I show how you can do just that for Breakout Joint Independence)
- Binary operations like Frankenstein or Stitching probably don’t fall out of an approach like that (they look a lot more like colimits
- Rules which a categorical approach trivializes like Factorization Transfer don’t particularly fall out of an approach like that
- You could probably get some kind of “ruleset completeness” result if you had a strong sense of what would constitute a complete ruleset—the paper I linked above looks to be trying to do this
...so maybe some approach where you start by listing off all the identities you want to have hold and then derive the full ruleset from those would work at least partially? I guess I’m not totally clear what you mean by “categorical axioms” here—there’s a fair amount that goes into a category; you’re setting down all the rules for a tiny toy mathematical universe and are relatively unconstrained in how you do so, apart from your desired semantics. I’m also not totally confident that I’ve answered your question.
Okay, so this is not what you care about? Maybe you are saying the following: Given two diagrams X,Y, we want to ask whether any distribution compatible with X is compatible with Y. We don’t ask whether the converse also holds. This is a certain asymmetric relation, rather than an equivalence.
Yes. Well… almost. Classically you’d care a lot more about the MEC, but I’ve gathered that that’s not actually true for latential Bayes nets. For those, we have some starting joint distribution J, which we care about a lot, and some diagram D_J which it factors over; alternatively, we have some starting set of conditionality requirements J, and some diagram D_J from among those that realize J. (The two cases are the same apart from the map that gives you actual numerical valuations for the states and transitions.)
We do indeed care about the asymmetric relation of “is every distribution compatible with X also compatible with Y”, but we care about it because we started with some X_J and transformed it such that at every stage, the resulting diagram was still compatible with J.

Lorxus Apr 8, 2025, 1:43 PM
LW: 3 AF: 1
0
AF
in reply to: Vanessa Kosoy’s comment on: Some Rules for an Algebra of Bayes Nets
that’s not a thing that really happens?
What is the thing that doesn’t happen? Reading the rest of the paragraph only left me more confused.
Somehow being able to derive all relevant string diagram rewriting rules for latential string diagrams, starting with some fixed set of equivalences? Maybe this is just a framing thing, or not answering your question, but I would expect to need to explicitly assume/include things like the Frankenstein rule—more generally, picking which additional rewriting rules you want to use/include/allow is something you do in order to get equivalence/reachability in the category to line up correctly with what you want that equvialence to mean—you could just as soon make those choices poorly and (e.g.) allow arbitrary deletions of states, or addition of sampling flow that can’t possibly respect causality; we don’t do that here because it wouldn’t result in string diagram behavior that matches what we want it to match.
I think I may have misunderstood, though, because that does sounds almost exactly like what I found to happen for the Joint Independence rule—you have some stated equivalence, and then you use string diagram rewriting rules common to all Markov category string diagrams in order to write the full form of the rule.
Simple form of the JI Rule—note the equivalences in the antecedent, which we can use to prove the consequent, not just enforce the equivalence!
we don’t quite care about Markov equivalence class
What do you mean by “Markov equivalence class”?
Two Bayes nets are of the same Markov equivalence class when they have precisely the same set of conditionality relations holding on them (and by extension, precisely the same undirected skeleton). You may recognize this as something that holds true when applicable for most of these rules, notably (sometimes) including the binary operations and notably excluding direct arrow-addition; additionally, this is something that applies for abstract Bayes nets just as well as concrete ones.

Lorxus Apr 7, 2025, 7:53 PM
LW: 3 AF: 2
0
AF
in reply to: Vanessa Kosoy’s comment on: Some Rules for an Algebra of Bayes Nets
Hi! I spent some time working on exactly this approach last summer at MATS, and since then have kept trying to work out the details. It’s been going slowly but parts of it have been going at all.
My take regarding your and @Alexander Gietelink Oldenziel’s comments below is—that’s not a thing that really happens? You pick the rules and your desired semantics first, and then you write down your manipulation rules to reflect those. Maybe it turns out that you got it wrong and there’s more (or fewer!) rules of derivation to write down, but as long as the rules you wrote down follow some very easy constraints, you can just… write down anything. Or decline to. (Also, from a quick read I’m fairly sure you need a little bit more than just “free cartesian category” to hold of your domain category if you want nice structures to hang state/kernel-labels on. You want to be able to distinguish a concrete Bayes net (like, with numbers and everything) from an abstract “Bayes net”, which is just a directed acyclic graph representing local conditionality structures—though we should note that usually people working with Bayes nets assume that we actually have trees, and not just DAGs!)
Maybe there’s arguments from things like mathematical elegance or desired semantics or problematizing cases to choose to include or exclude some rule, or formulate a rule in one or another way, but ultimately you get to add structure to a category as suits you.
- Can we derive a full classification of such rules?
Probably there’s no full classification, but I think that a major important point (which I should have just written up quickly way way long ago) is that we don’t quite care about Markov equivalence class—only about whether any given move takes us to a diagram still compatible with the joint distribution/factorization that we started with.
- Is there a category-theoretic story behind the rules? Meaning, is there a type of category for which Bayes nets are something akin to string diagrams and the rules follow from the categorical axioms?
In my understanding there’s a pretty clear categorical story—some of which are even just straight trivial! - to most of the rules. They make good sense in the categorical frame; some of them make so much sense that they’re the kind of thing string diagrams don’t even bother to track. I’m thinking of Factorization Transfer here, though the one major thing I’ve managed to get done is that the Joint Independence rule actually follows as a quick technical lemma from its assumptions/setup, rather than having to be assumed.
There’s noncanonical choices getting made in the two “binary operation” rules, but even then it seems to me like the fact that we only ever apply that operation to two diagrams of the same “flavor” means we should likely only ever get back another diagram of the same “flavor”; apart from that, there’s a strong smell of colimitness to both of those rules, which has kept me mulling over whether there’s a useful “limitish” sort of rule that John and David both missed. Underlying this is my suspicion that we might get a natural preorder structure on latent Bayes nets/string diagrams as the pullback of the natural lattice on Markov equivalence classes, where one MEC is above another if it has strictly more conditionality relationships in it—i.e. the unique minimal element of the lattice is “everything is independent”, and there’s lots of maximal elements, and all our rules need to keep us inside the preimage of the upset of our starting MEC.
Which is to say: yes, yes there totally is such a type of category, but existing work mostly stopped at “Bayes nets akin to string diagrams”, although I’ve seen some non-me work lately that looks to be attacking similar problems.

Lorxus Apr 4, 2025, 10:52 PM
1 point
0
in reply to: LoganStrohl’s comment on: BrienneYudkowsky’s Shortform
Did you ever get back to reading this? I think I got some very different things out of it when I read through! (And @whatstruekittycat will talk your ear off about it, among other topics.)

Lorxus Apr 4, 2025, 12:57 AM
5 points
2
on: Steelmanning Divination
I think you maybe miss an entire branch of the tech-tree here I consider important—the bit about the Lindy case of divination with a coin-flip and checking your gut. It doesn’t stop at a single bit in my experience; it’s something you can use more generally to get your own read on some situation much less filtered by masking-type self-delusion. At the absolute least, you can get a “yes/no/it’s complicated” out of it pretty easily with a bit more focusing!
I claim that divination^[1] is specifically a good way for routing around the worried self-deception and yeah-sure-whatever daily frames and getting at how you really feel about something you’re clearly uncertain-and-high-arousal enough about to Do A Divination about.
As a maybe-tangent, a friend of mine has a self-designed divination method I like a lot but haven’t managed to fully learn how to do; what I like about it is its compositionality/modularity, where you pull a handful (3-5 IIRC) of symbolic rune-type things from a bag and toss them onto a surface, such that both the divination-tiles you pulled out and their relative-location/absolute-location/orientation/pointing-with-respect-to-each-other all matter. I don’t think I have a maximally good sense of why I like it so much past the obvious factors of “fun system/novelty/a friend made this” and “fine-grained expressive power” and “combinatorial explosion power”, though.
For my own part I like to use coins that I’ve made for other reasons for this purpose. They’re not fair coins; that’s the fun part.
1. ^
  In the explicit epistemic frame of “we are getting the bits of interest from some internal mental process to use for planning/self-check purposes”, mind you!

Lorxus Mar 24, 2025, 6:52 PM
3 points
0
in reply to: lsusr’s comment on: Proselytizing
Got it, thanks. I’ll see if I can figure out who that was or where to find that claim. Cheers.

Lorxus Mar 24, 2025, 4:28 PM
1 point
0
on: Proselytizing
Maybe this is the right place to ask/discuss this, and maybe not—if it’s not; say so and I’ll stop.
IIRC you (or maybe someone once mentioned hearing about people who try to [experience the first jhana]^[1] and then feeling pain as a result, and that you didn’t really understand why that happened. There was maybe also a comment about “don’t do that, that sounds like you were doing it wrong”.
After some time spent prodding at myself and pulling threads and seeing where they lead… I am not convinced that they were doing it wrong at all. There’s a kind of way you can end up where the application of that kind of comfort/pleasure/positive-valence is, of itself, painful/aversive/immiserating, if not necessarily enduringly so. Reframing it
I don’t have a full explicit model for it, so here’s some metaphors that hopefully collectively shed light:
- Hunger beyond reason, hunger for weeks, hunger to the point of starvation. A rich and lavish meal set before you, of all your favorite foods and drinks, prepared expertly. A first bite—overwhelming; so perfect and so intense. You toy with the idea of eating nothing more and find you can neither eat nor decline—at least not comfortably. You gorge yourself and die of refeeding syndrome.
- Dreams of your childhood home, of the forests around it, of the sparkling beauty of the night sky. The building was knocked down years ago, the forest cut, the sky bleached with light pollution, all long after you moved away anyway.
- Like the itch/pain of a healing wound, or of a limb fallen asleep, or an amputated limb. Like internal screaming or weeping, suddenly given voice.
- Like staring at something dazzlingly bright and incomparably precious, even coveted; especially one that you can’t touch or even reach—a sapphire the size of your fist, say, or the sun. What would you even do with those, really, if you could grab them?
1. ^
  Not sure if my terminology is correct here—I’m talking about doing the meditation/mental-action process itself. You know, the one which causes you tons of positive valence in a way you like but don’t want.

Lorxus Mar 24, 2025, 2:22 PM
4 points
1
on: Lorxus’s Shortform
Here’s a game-theory game I don’t think I’ve ever seen explicitly described before: Vicious Stag Hunt, a two-player non-zero-sum game elaborating on both Stag Hunt and Prisoner’s Dilemma. (Or maybe Chicken? It depends on the obvious dials to turn. This is frankly probably a whole family of possible games.)

The two players can pick from among 3 moves: Stag, Hare, and Attack.

Hunting stag is great, if you can coordinate on it. Playing Stag costs you 5 coins, but if the other player also played Stag, you make your 5 coins back plus another 10.

Hunting hare is fine, as a fallback. Playing Hare costs you 1 coin, and assuming no interference, makes you that 1 coin back plus another 1.

But to a certain point of view, the richest targets are your fellow hunters. Preparing to Attack costs you 2 coins. If the other player played Hare, they escape you, barely recouping their investment (0 payoff), and you get nothing for your boldness. If they played Stag, though, you can backstab them right after securing their aid, taking their 10 coins of surplus destructively, costing them 10 coins on net. Finally, if you both played Attack, you both starve for a while waiting for the attack, you heartless fools. Your payoffs are symmetric, though this is one of the most important dials to turn: if you stand to lose less in such a standoff than you would by getting suckered, then Attack dominates Stag. My scratchpad notes have payoffs at (-5, −5), for instance.

To resummarize the payoffs:
- (H, H) = (1, 1)
- (H, S) = (1, −5)
- (S, S) = (10, 10)
- (H, A) = (0, −2)
- (S, A) = (-10(*), 20)
- (A, A) = (-n, -n); n <(=) 10(*) → A >(=) S
So what happens? Disaster! Stag is dominated, so no one plays it, and everyone converges to Hare forever.

And what of the case where n > 10? While initially I’d expected a mixed equilibrium, I should have expected the actual outcome: the sole Nash equilibrium is still the pure all-Hare strategy—after all, we’ve made Attacking strictly worse than in the previous case! (As given by https://cgi.csc.liv.ac.uk/~rahul/bimatrix_solver/ ; I tested n = 12.)

Lorxus Mar 23, 2025, 4:49 PM
25 points
2
on: Lorxus’s Shortform
A snowclone summarizing a handful of baseline important questions-to-self: “What is the state of your X, and why is that what your X’s state is?” Obviously also versions that are less generally and more naturally phrased, that’s just the most obviously parametrized form of the snowclone.
Classic(?) examples:
”What do you (think you) know, and why do you (think you) know it?” (X = knowledge/belief)
”What are you doing, and why are you doing it?” (X = action(-direction?)/motivation?)
Less classic examples that I recognized or just made up:
”How do you feel, and why do you feel that way?” (X = feelings/emotions)
”What do you want, and why do you want it?” (X = goal/desire)
”Who do you know here, and how do you know them?” (X = social graph?)
”What’s the plan here, and what are you hoping to achieve by that plan?” (X = plan)

Lorxus Mar 19, 2025, 7:03 PM
10 points
5
on: The principle of genomic liberty
I think this post is pretty cool, and represents good groundwork on sticky questions of bioethics and the principles that should underpin them that most people don’t think about very hard. Thanks for writing it.

Lorxus Mar 19, 2025, 6:37 PM
1 point
0
in reply to: Gurkenglas’s comment on: Lorxus’s Shortform
The phrasing I got from the mentor/research partner I’m working with is pretty close to the former but closer in attitude and effective result to the latter. Really, the major issue is that string diagrams for a flavor of category and commutative diagrams for the same flavor of category are straight-up equivalent, but explicitly showing this is very very messy, and even explicitly describing Markov categories—the flavor of category I picked as likely the right one to use, between good modelling of Markov kernels and their role doing just that for causal theories (themselves the categorification of “Bayes nets up to actually specifying the kernels and states numerically”) - is probably too much to put anywhere in a post but an appendix or the like.

...if there is no available action like snapping a photo that takes less time than writing the reply I’m replying to did...

There is not, but that’s on me. I’m juggling too much and having trouble packaging my research in a digestible form. Precarious/lacking funding and consequent binding demands on my time really don’t help here either. I’ll add you to the long long list of people who want to see a paper/post when I finally complete one.

I guess a major blocker for me is—I keep coming back to the idea that I should write the post as a partially-ordered series of posts instead. That certainly stands out to me as the most natural form for the information, because there’s three near-totally separate branches of context—Bayes nets, the natural latent/abstraction agenda, and (monoidal category theory/)string diagrams—of which you need to somewhat understand some pair in order to understand major necessary background (causal theories, motivation for Bayes net algebra rules, and motivation for string diagram use), and all three to appreciate the research direction properly. But I’m kinda worried that if I start this partially-ordered lattice of posts, I’ll get stuck somewhere. Or run up against the limits of what I’ve already worked out yet. Or run out of steam with all the writing and just never finish. Or just plain “no one will want to read through it all”.

Lorxus Mar 19, 2025, 2:46 PM
1 point
0
in reply to: Gurkenglas’s comment on: Lorxus’s Shortform
I guess? I mean, there’s three separate degrees of “should really be kept contained”-ness here:
- Category theory → string diagrams, which pretty much everyone keeps contained, including people who know the actual category theory
- String diagrams → Bayes nets, which is pretty straightforward if you sit and think for a bit about the semantics you accept/are given for string diagrams generally and maybe also look at a picture of generators and rules—not something anyone needs to wrap up nicely but it’s also a pretty thin
- [Causal theory/Bayes net] string diagrams → actual statements about (natural) latents, which is something I am still working on; it’s turning out to be pretty effortful to grind through all the same transcriptions again with an actually-proof-usable string diagram language this time. I have draft writeups of all the “rules for an algebra of Bayes nets”—a couple of which have turned out to have subtleties that need working out—and will ideally be able to write down and walk through proofs entirely in string diagrams while/after finishing specifications of the rules.
So that’s the state of things. Frankly I’m worried and generally unhappy about the fact that I have a post draft that needs restructuring, a paper draft that needs completing, and a research direction to finish detailing, all at once. If you want some partial pictures of things anyway all the same, let me know.

Lorxus Mar 19, 2025, 2:32 PM
3 points
0
on: Equations Mean Things
Not much to add apart from “this is clean and really good, thanks!”.

Lorxus Mar 18, 2025, 9:30 PM
4 points
0
on: Lorxus’s Shortform
I promise I am still working on working out all the consequences of the string diagram notation for latential Bayes nets, since the guts of the category theory are all fixed (and can, as a mentor advises me, be kept out of the public eye as they should be). Things can be kept (basically) purely in terms of string diagrams. In whatever post I write, they certainly will be.

I want to be able to show that isomorphism of natural latents is the categorical property I’m ~97% sure it is (and likewise for minimal and maximal latents). I need to sit myself down and at least fully transcribe the Fundamental Theorem of Latents in preparation for supplying the proof to that.

Mostly I’m spending a lot of time on a data science bootcamp and an AISC track and taking care of family and looking for work/funding and and and.

Lorxus Mar 18, 2025, 9:10 PM
1 point
0
on: Lorxus’s Shortform
Because RLHF works, we shouldn’t be surprised when AI models output wrong answers which are specifically hard for humans to distinguish from a right answer.

This observably (seems like it) generalizes to all humans, instead of (say) it being totally trivial somehow to train an AI on feedback only from some strict and distinguished subset of humanity such that any wrong answers it produced could be easily spotted by the excluded humans.

Such wrong answers which look right (on first glance) also observably exist, and we should thus expect that if there’s anything like a projection-onto-subspace going on here, our “viewpoint” for the projection, given any adjudicating human mind, is likely all clustered in some low-dimensional subspace of all possible viewpoints and maybe even just around a single point.

This is why I’d agree that RLHF was so specifically a bad tradeoff in capabilities improvement vs safety/desirability outcomes but still remain agnostic as to the absolute size of that tradeoff.

Lorxus Mar 16, 2025, 6:51 PM
1 point
0
on: Report & retrospective on the Dovetail fellowship
Sent you an admonymous.

Lorxus Mar 16, 2025, 6:20 PM
3 points
0
on: Lorxus’s Shortform
(Random thought I had and figured this was the right place to set it down:) Given how centally important token-based word embeddings as to the current LLM paradigm, how plausible is it that (put loosely) “doing it all in Chinese” (instead of English) is actually just plain a more powerful/less error-prone/generally better background assumption?

Associated helpful intuition pump: LLM word tokenization is like a logographic writing system, where each word corresponds to a character of the logography. There need be no particular correspondence between the form of the token and the pronunciation/”alphabetical spelling”/other things about the word, though it might have some connection to the meaning of the word—and it often makes just as little sense to be worried about the number of grass radicals in “草莓” as it does to worry about the number of r’s in a “strawberry” token.

(And yes, I am aware that in Mandarin Chinese, there’s lots of multi-character words and expressions!)

Lorxus Mar 16, 2025, 5:06 PM
3 points
0
in reply to: Garrett Baker’s comment on: johnswentworth’s Shortform
As someone who does both data analysis and algebraic topology, my take is that TDA showed promise but ultimately there’s something missing such that it’s not at full capacity. Either the formalism isn’t developed enough or it’s being consistently used on the wrong kinds of datasets. Which is kind of a shame, because it’s the kind of thing that should work beautifully and in some cases even does!

Lorxus Mar 15, 2025, 11:25 PM
3 points
0
in reply to: danielechlin’s comment on: johnswentworth’s Shortform
I imagine it’s something like “look for things that are notably absent, when you would expect them to have been found if there”?