Lorxus

Karma: 322

Mathematician, alignment researcher, doctor. Reach out to me on Discord and tell me you found my profile on LW if you’ve got something interesting to say; you have my explicit permission to try to guess my Discord handle if so. You can’t find my old abandoned-for-being-mildly-infohazardously-named LW account but it’s from 2011 and has 280 karma.

A Lorxus Favor is worth (approximately) one labor-day’s worth of above-replacement-value specialty labor, given and received in good faith, and used for a goal approximately orthogonal to one’s desires, and I like LessWrong because people here will understand me if I say as much.

Apart from that, and the fact that I am under no NDAs, including NDAs whose existence I would have to keep secret or lie about, you’ll have to find the rest out yourself.

Lorxus Mar 19, 2025, 7:03 PM
9 points
4
on: The principle of genomic liberty
I think this post is pretty cool, and represents good groundwork on sticky questions of bioethics and the principles that should underpin them that most people don’t think about very hard. Thanks for writing it.

Lorxus Mar 19, 2025, 6:37 PM
1 point
0
in reply to: Gurkenglas’s comment on: Lorxus’s Shortform
The phrasing I got from the mentor/research partner I’m working with is pretty close to the former but closer in attitude and effective result to the latter. Really, the major issue is that string diagrams for a flavor of category and commutative diagrams for the same flavor of category are straight-up equivalent, but explicitly showing this is very very messy, and even explicitly describing Markov categories—the flavor of category I picked as likely the right one to use, between good modelling of Markov kernels and their role doing just that for causal theories (themselves the categorification of “Bayes nets up to actually specifying the kernels and states numerically”) - is probably too much to put anywhere in a post but an appendix or the like.

...if there is no available action like snapping a photo that takes less time than writing the reply I’m replying to did...

There is not, but that’s on me. I’m juggling too much and having trouble packaging my research in a digestible form. Precarious/lacking funding and consequent binding demands on my time really don’t help here either. I’ll add you to the long long list of people who want to see a paper/post when I finally complete one.

I guess a major blocker for me is—I keep coming back to the idea that I should write the post as a partially-ordered series of posts instead. That certainly stands out to me as the most natural form for the information, because there’s three near-totally separate branches of context—Bayes nets, the natural latent/abstraction agenda, and (monoidal category theory/)string diagrams—of which you need to somewhat understand some pair in order to understand major necessary background (causal theories, motivation for Bayes net algebra rules, and motivation for string diagram use), and all three to appreciate the research direction properly. But I’m kinda worried that if I start this partially-ordered lattice of posts, I’ll get stuck somewhere. Or run up against the limits of what I’ve already worked out yet. Or run out of steam with all the writing and just never finish. Or just plain “no one will want to read through it all”.

Lorxus Mar 19, 2025, 2:46 PM
1 point
0
in reply to: Gurkenglas’s comment on: Lorxus’s Shortform
I guess? I mean, there’s three separate degrees of “should really be kept contained”-ness here:
- Category theory → string diagrams, which pretty much everyone keeps contained, including people who know the actual category theory
- String diagrams → Bayes nets, which is pretty straightforward if you sit and think for a bit about the semantics you accept/are given for string diagrams generally and maybe also look at a picture of generators and rules—not something anyone needs to wrap up nicely but it’s also a pretty thin
- [Causal theory/Bayes net] string diagrams → actual statements about (natural) latents, which is something I am still working on; it’s turning out to be pretty effortful to grind through all the same transcriptions again with an actually-proof-usable string diagram language this time. I have draft writeups of all the “rules for an algebra of Bayes nets”—a couple of which have turned out to have subtleties that need working out—and will ideally be able to write down and walk through proofs entirely in string diagrams while/after finishing specifications of the rules.
So that’s the state of things. Frankly I’m worried and generally unhappy about the fact that I have a post draft that needs restructuring, a paper draft that needs completing, and a research direction to finish detailing, all at once. If you want some partial pictures of things anyway all the same, let me know.

Lorxus Mar 19, 2025, 2:32 PM
3 points
0
on: Equations Mean Things
Not much to add apart from “this is clean and really good, thanks!”.

Lorxus Mar 18, 2025, 9:30 PM
4 points
0
on: Lorxus’s Shortform
I promise I am still working on working out all the consequences of the string diagram notation for latential Bayes nets, since the guts of the category theory are all fixed (and can, as a mentor advises me, be kept out of the public eye as they should be). Things can be kept (basically) purely in terms of string diagrams. In whatever post I write, they certainly will be.

I want to be able to show that isomorphism of natural latents is the categorical property I’m ~97% sure it is (and likewise for minimal and maximal latents). I need to sit myself down and at least fully transcribe the Fundamental Theorem of Latents in preparation for supplying the proof to that.

Mostly I’m spending a lot of time on a data science bootcamp and an AISC track and taking care of family and looking for work/funding and and and.

Lorxus Mar 18, 2025, 9:10 PM
1 point
0
on: Lorxus’s Shortform
Because RLHF works, we shouldn’t be surprised when AI models output wrong answers which are specifically hard for humans to distinguish from a right answer.

This observably (seems like it) generalizes to all humans, instead of (say) it being totally trivial somehow to train an AI on feedback only from some strict and distinguished subset of humanity such that any wrong answers it produced could be easily spotted by the excluded humans.

Such wrong answers which look right (on first glance) also observably exist, and we should thus expect that if there’s anything like a projection-onto-subspace going on here, our “viewpoint” for the projection, given any adjudicating human mind, is likely all clustered in some low-dimensional subspace of all possible viewpoints and maybe even just around a single point.

This is why I’d agree that RLHF was so specifically a bad tradeoff in capabilities improvement vs safety/desirability outcomes but still remain agnostic as to the absolute size of that tradeoff.

Lorxus Mar 16, 2025, 6:51 PM
1 point
0
on: Report & retrospective on the Dovetail fellowship
Sent you an admonymous.

Lorxus Mar 16, 2025, 6:20 PM
3 points
0
on: Lorxus’s Shortform
(Random thought I had and figured this was the right place to set it down:) Given how centally important token-based word embeddings as to the current LLM paradigm, how plausible is it that (put loosely) “doing it all in Chinese” (instead of English) is actually just plain a more powerful/less error-prone/generally better background assumption?

Associated helpful intuition pump: LLM word tokenization is like a logographic writing system, where each word corresponds to a character of the logography. There need be no particular correspondence between the form of the token and the pronunciation/”alphabetical spelling”/other things about the word, though it might have some connection to the meaning of the word—and it often makes just as little sense to be worried about the number of grass radicals in “草莓” as it does to worry about the number of r’s in a “strawberry” token.

(And yes, I am aware that in Mandarin Chinese, there’s lots of multi-character words and expressions!)

Lorxus Mar 16, 2025, 5:06 PM
3 points
0
in reply to: Garrett Baker’s comment on: johnswentworth’s Shortform
As someone who does both data analysis and algebraic topology, my take is that TDA showed promise but ultimately there’s something missing such that it’s not at full capacity. Either the formalism isn’t developed enough or it’s being consistently used on the wrong kinds of datasets. Which is kind of a shame, because it’s the kind of thing that should work beautifully and in some cases even does!

Lorxus Mar 15, 2025, 11:25 PM
3 points
0
in reply to: danielechlin’s comment on: johnswentworth’s Shortform
I imagine it’s something like “look for things that are notably absent, when you would expect them to have been found if there”?

Lorxus Nov 16, 2024, 6:52 PM
1 point
0
on: Internal music player: phenomenology of earworms
Do you know when you started experiencing having an internal music player? I recall that that started for me when I was about 6. Also, do you know whether you can deliberately pick a piece of music, or other nonmusical sonic experiences, to playback internally? Can you make them start up from internal silence? Under what conditions can you make them stop? Do you ever experience long stretches where you have no internal music at all?

Lorxus Nov 16, 2024, 5:09 PM
7 points
0
in reply to: Chipmonk’s comment on: Ayn Rand’s model of “living money”; and an upside of burnout
Sure—I can believe that that’s one way a person’s internal quorum can be set up. In other cases, or for other reasons, they might be instead set up to demand results, and evaluate primarily based on results. And that’s not great or necessarily psychologically healthy, but then the question becomes “why do some people end up one way and other people the other way?” Also, there’s the question of just how big/significant the effort was, and thus how big of an effective risk the one predictor took. Be it internal to one person or relevant to a group of humans, a sufficiently grand-scale noble failure will not generally be seen as all that noble (IME).

Lorxus Nov 16, 2024, 4:49 PM
8 points
2
in reply to: Chipmonk’s comment on: Ayn Rand’s model of “living money”; and an upside of burnout
This makes some interesting predictions re: some types of trauma: namely, that they can happen when someone was (probably even correctly!) pushing very hard towards some important goal, and then either they ran out of fuel just before finishing and collapsed, or they achieved that goal and then—because of circumstances, just plain bad luck, or something else—that goal failed to pay off in the way that it usually does, societally speaking. In either case, the predictor/pusher that burned down lots of savings in investment doesn’t get paid off. This is maybe part of why “if trauma, and help, you get stronger; if trauma, and no help, you get weaker”.

Lorxus Nov 11, 2024, 9:10 PM
1 point
0
on: D&D.Sci Coliseum: Arena of Data Evaluation and Ruleset
I didn’t enjoy this one as much, but that’s likely down to not having had the time/energy to spend on thinking this through deeply. That said… I did not in fact enjoy it as much and I mostly feel like garbage for having done literally worse than chance, and I feel like it probably would have been better if I hadn’t participated at all.

Lorxus Nov 7, 2024, 4:10 AM
3 points
0
in reply to: johnswentworth’s comment on: Some Rules for an Algebra of Bayes Nets
Let me see if I’ve understood point 3 correctly here. (I am not convinced I have actually found a flaw, I’m just trying to reconcile two things in my head here that look to conflict, so I can write down a clean definition elsewhere of something that matters to me.)
$P$ factors over $G$ . In $G$ , $X_{j}, X_{k}$ were conditionally independent of each other, given $X_{p a^{G} (k)}$ . Because $P$ factors over $G$ and because in $G$ , $X_{j}, X_{k}$ were conditionally independent of each other, given $X_{p a^{G} (k)}$ , we can very straightforwardly show that $P$ factors over $G^{'}$ , too. This is the stuff you said above, right?
But if we go the other direction, assuming that some arbitrary $P^{'}$ factors over $G^{'}$ , I don’t think that we can then still derive that $P^{'}$ factors over $G$ in full generality, which was what worried me. But that break of symmetry (and thus lack of equivalence) is… genuinely probably fine, actually—there’s no rule for arbitrarily deleting arrows, after all.
That’s cleared up my confusion/worries, thanks!

Lorxus Nov 6, 2024, 9:55 PM
3 points
0
on: Some Rules for an Algebra of Bayes Nets
We’ll refer to these as “Bookkeeping Rules”, since they feel pretty minor if you’re already comfortable working with Bayes nets. Some examples:
- We can always add an arrow to a diagram (assuming it doesn’t introduce a loop), and the approximation will get no worse.
Here’s something that’s kept bothering me on and off for the last few months: This graphical rule immediately breaks Markov equivalence. Specifically, two DAGs are Markov-equivalent only if they share an (undirected) skeleton. (Lemma 6.1 at the link.)
If the major/only thing we care about here regarding latential Bayes nets is that our Grand Joint Distribution $P [X_{G}]$ factorize over (that is, satisfy) our DAG $G$ (and all of the DAGs we can get from it by applying the rules here), then by Thm 6.2 in the link above, $P$ is also globally/locally Markov wrt $G$ . This holds even when $P [X_{G}] > 0$ is not guaranteed for some of the possible joint states in $X_{G}$ , unlike Hammersley-Clifford would require.
That in turn means that (Def 6.5) there’s some distributions $P$ can be such that $P [X_{G}]$ factors over $G$ , but not $G^{'} = G + a n o n l o o p y e x t r a a r r o w$ (where $G^{'}$ trivially has the same vertices as $G$ does); specifically, because $G, G^{'}$ don’t (quite) share a skeleton, they can’t be Markov-equivalent, and because they aren’t Markov-equivalent, $P$ no longer needs to be (locally/globally) Markov wrt $G^{'}$ (and in fact there must exist some $P$ which explicitly break this), and because of that, such $P$ need not factor over $G^{'}$ . Which I claim we should not want here, because (as always) we care primarily about preserving which joint probability distributions factorize over/satisfy which DAGs, and of course we probably don’t get to pick whether our $P$ is one of the ones where that break in the chain of logic matters.

Lorxus Oct 22, 2024, 6:49 PM
5 points
0
on: D&D Sci Coliseum: Arena of Data
I’m going to start by attacking this a little on my own before I even look much at what other people have done.
Some initial observations from the SQL+Python practice this gave me a good excuse to do:
- Adelon looks to have rough matchups against Elf Monks. Which we don’t have. They are however soft to even level 3-4 challengers sometimes. Maybe Monks and/or Fencers have an edge on Warriors?
- Bauchard seems to have particularly strong matchups against other Knights, so we don’t send Velaya there. They seem a little soft to Monks and to Dwarf Ninjas and especially to Knights, so maybe Zelaya? Boots should help here.
- Cadagal has precious few defeats, but one of them might be to a level 2(!) Human Warrior with fancy +3 Gauntlets. Though it seems like there’s a lot of combats where some Cadagal-like fighter has +4 Boots instead? Not sure if that’s the same guy.
  And on that note, the max level is 7, and the max bonus for Boots and Gauntlets both is +4.
  Max Boots (+4) is always on a level 7 Elf Ninja with +3 Gauntlets (but disappears altogether most of the way through the dataset).
  Max Gauntlets (+4) is on either a level 7 Dwarf Monk who upgraded from +1 Boots to +3 Boots halfway through, or else there’s two of them. Thankfully we’re not facing them.
- Deepwrack poses problems. They have just as few defeats, and one of them even contradicts the ordering I derived below! Ninjas are meant to lose to Monks. Maybe the speed matters a lot in that case?
- It looks like a strict advantage in level or gear—holding all else constant—means you win every time. If everything is totally identical, you win about half the time. (Which seems obvious but worth checking.)
- Looking through upsets—bouts where the classes are different, the losing fighter had at least 2 levels on the winner, and the loser’s gear was no better than the winner’s—we generally see that:
  Fencers beat Monks and Rangers and lose to Knights, Ninjas, and Warriors
  Knights beat Fencers and Ninjas, tie(???) with Monks and Warriors, and lose (weakly) to Rangers
  Monks beat Ninjas, Rangers, and maybe Warriors, tie (?) with Knights, and lose to Fencers
  Ninjas beat Fencers and (weakly) Rangers, and lose to Knights, Monks, and Warriors
  Rangers beat Knights (weakly), Ninjas, and Warriors, tie with Fencers, and lose to Monks
  Warriors beat Fencers, Ninjas, tie(?) with Knights, and lose to Rangers and maybe Monks
So my current best guess (pending understanding which gear is best for which class/race) is:
Willow v Adelon, Varina v Bauchard, Xerxes v Cadagal, Yalathinel v Deepwrack.
If I had to guess what gear to give to who: Warrior v Knight is a rough matchup, so Varina’s going to need the help; the rest of my assignments are based thus far on ~vibes~ for whether speed or power will matter more for the class. Thus:
Willow gets +2 Boots and +1 Gauntlets, Varina gets +4 Boots and +3 Gauntlets, Xerxes gets +1 Boots and +2 Gauntlets, and Yalathinel gets +3 Boots.
Some theories I need to test:
- Race affects how good you are at a class. Elves might be best at rangering, say.
- Race and/or class affect how much benefit you get out of boots and/or gauntlets. Being a warrior might mean you get full benefit from gauntlets but none from boots.
- Color might affect how well classes do. Ninjas wearing red might win way less often.
  The color does not actually seem to affect ninjas all that much if at all − 6963 vs 6762 wins. Could still be a tiebreaker?
  Color doesn’t affect things much overall either: 40136 vs 39961 wins.
- There’s some rank-ordering of class+race+level matchups, maybe an additive one.
  Alternatively there could be some nontransitive thing going on with tiebreaks sometimes from levels, races, and gear?
  On further reflection that totally seems to be what’s going on here.
  Maybe there’s something about the matchup ordering being sorted over (race, class)? D’s loss (as a L6 Dwarf Monk) to a L4 Dwarf Ninja is… unexpected to say the least!
Wild speculation:
- If you [use the +4 Boots in combat and beat Cadagal then they’ll know you were] responsible [for] ????? ?????? [Boots from his/her/the] House. [You will gain its] lasting enmity, [and] [people? will?] ???????? ???? ??? ???? ?? ??? ???? ??????? ?? ?? ????? ?? ????????? ?? ??? [upon] your honor [if] ????????? ???? ?? ???? ??? ???? ??? ??? friendship ???? ?? ??? ???? ?? ??? ?????? ??????? ?? ?? ?????.
  So maybe we’re OK to use the +4 Boots as long as it’s not against Cadagal?
  No idea how to even guess at what’s going on in that second sentence apart from “bad things will happen and everyone will hate you, you dirty thief”.

Lorxus Aug 29, 2024, 3:14 PM
3 points
0
on: Why Large Bureaucratic Organizations?
I’m gonna leave my thoughts on the ramifications for academia, where a major career step is to repeatedly join and leave different large bureaucratic organizations for a decade, as an exercise to the reader.

Like, in a world where the median person is John Wentworth (“Wentworld”), I’m pretty sure there just aren’t large organizations of the sort our world has.

I have numerous thoughts on how Lorxusverse Polity handles this problem but none of it is well-worked out enough to share. In sum though: Probably cybernetics (in the Beer sense) got discovered way earlier and actually ever used as stated and that was that, no particular need for dominance-status as glue or desire for it as social-good. (We’d be way less social overall, though, too, and less likely to make complex enduring social arrangements. There would be careful Polity-wide projects for improving social-contact and social-nutrition. They would be costly and weird. Whether that’s good or bad on net, I can’t say.)

Lorxus Aug 28, 2024, 3:22 PM
1 point
0
in reply to: M. Y. Zuo’s comment on: Shifting Headspaces—Transitional Beast-Mode
Sure, but you obviously don’t (and can’t even in principle) turn that up all the way! The key is to make sure that that mode still exists and that you don’t simply amputate and cauterize it.

Lorxus Aug 28, 2024, 3:16 PM
1 point
0
in reply to: TsviBT’s comment on: Wei Dai’s Shortform
[2.] maybe one could go faster by trying to more directly cleave to the core philosophical problems.
...
An underemphasized point that I should maybe elaborate more on: a main claim is that there’s untapped guidance to be gotten from our partial understanding—at the philosophical level and for the philosophical level. In other words, our preliminary concepts and intuitions and propositions are, I think, already enough that there’s a lot of progress to be made by having them talk to each other, so to speak.
OK but what would this even look like?\gen
Toss away anything amenable to testing and direct empirical analysis; it’s all too concrete and model-dependent.
Toss away mathsy proofsy approaches; they’re all too formalized and over-rigid and can only prove things from starting assumptions we haven’t got yet and maybe won’t think of in time.
Toss away basically all settled philosophy, too; if there were answers to be had there rather than a few passages which ask correct questions, the Vienna Circle would have solved alignment for us.
What’s left? And what causes it to hang together? And what causes it not to vanish up its own ungrounded self-reference?