cubefox

Karma: 1,438

cubefox 27 Jan 2025 15:38 UTC
2 points
0
on: Logits, log-odds, and loss for parallel circuits
As you say, the addition of logits is equivalent to the multiplication of odds. And odds are related to probability with $o = \frac{p}{1 - p}$ . (One can view them as different scalings of the same quantity. Probabilities have the range $[0, 1]$ , odds have the range $[0, + \infty]$ , and logits have the range $[- \infty, + \infty]$ .)

Now a well-known fact about multiplication of probabilities $P$ is this:
- $P (A) \times P (B) = P (A \land B)$ , when $A$ and $B$ are independent.
But there is a similar fact about the multiplication of odds $O$ , though not at all well-known:
- $O (A) \times O (B) = O (A \land B ∣ A \leftrightarrow B)$ , when $A$ and $B$ are independent.
That is, multiplying the odds of two independent events/propositions gives you the probability of their conjunction, given that their biconditional is true, i.e. given that they have the same truth values / that they either both happen or both don’t happen.

Perhaps this yields some more insight in how to interpret practical logit addition.

cubefox 26 Jan 2025 10:59 UTC
4 points
2
in reply to: yrimon’s comment on: yrimon’s Shortform
Yes. Current reasoning models like DeepSeek-R1 rely on verified math and coding data sets to calculate the reward signal for RL. It’s only a side-effect if they also get better at other reasoning tasks, outside math and programming puzzles. But in theory we don’t actually need strict verifiability for a reward signal, only your much weaker probabilistic condition. In the future, a model could check the goodness of it’s own answers. At which point we would have a self-improving learning process, which doesn’t need any external training data for RL.

And it is likely that such a probabilistic condition works on many informal tasks. We know that checking a result is usually easier than coming up with the result, even outside exact domains. E.g. it’s much easier to recognize a good piece of art than to create one. This seems to be a fundamental fact about computation. It is perhaps a generalization of the apparent fact that NP problems (with quickly verifiable solutions) cannot in general be reduced to P problems (which are quickly solvable).

cubefox 25 Jan 2025 12:25 UTC
2 points
0
in reply to: someonewrongonthenet’s comment on: To become more rational, rinse your left ear with cold water
Apparently yes

cubefox 21 Jan 2025 7:53 UTC
1 point
−1
on: Chance is in the Map, not the Territory
A problem with subjective probability is that it ignores any objective fact which would make one probability “better” or “more accurate” than the other. Someone could believe a perfectly symmetrical coin has a 10% chance of coming up heads, even though such a coin is physically impossible.

The concept of subjective probability is independent of any fact about objectively existing symmetries and laws. It also ignores physical dispositions, called propensities, which is like denying that a vase is breakable because this would, allegedly, be like positing a “mysterious force” which makes it true that the vase would break if it dropped.

Subjective probability is only a measure of degree of belief, not of what a “rational” degree of belief would be, and neither is it a measure of ignorance, of how much evidence someone has about something being true or false.

It is also semantically implausible. It is perfectly valid to say “I thought the probability was low, but it was actually high. I engaged in wishful thinking and ignored the evidence I had.” But with subjective probability this would be a contradiction, it would be equivalent to saying “My degree of belief was low, but it was actually high”. That’s not what the first sentence actually expresses.

cubefox 8 Dec 2024 1:23 UTC
6 points
2
on: Algebraic Linguistics
Related: In the equation $y = a x + b$ , the values of all four variables are unknown, but x and y seem to be more unknown (more variable?) than a and b. It’s not clear what the difference is exactly.

cubefox 8 Dec 2024 0:31 UTC
8 points
4
on: Understanding Shapley Values with Venn Diagrams
Explaining the Shapley value in terms of the “synergies” (and the helpful split in the Venn diagram) makes much more intuitive sense than the more complex normal formula without synergies, which is usually just given without motivation. That being said, it requires first computing the synergies, which seems somewhat confusing for more than three players. The article itself doesn’t mention the formula for the synergy function, but Wikipedia has it.

cubefox 5 Dec 2024 5:23 UTC
4 points
0
on: You are not too “irrational” to know your preferences.
Let me politely disagree with this post. Yes, often desires (“wants”) are neither rational nor irrational, but that’s far from always the case. Let’s begin with this:

But the fundamental preferences you have are not about rationality. Inconsistent actions can be irrational if they’re self-defeating, but “inconsistent preferences” only makes sense if you presume you’re a monolithic entity, or believe your “parts” need to all be in full agreement all the time… which I think very badly misunderstands how human brains work.

In the above quote you could simply replace “preferences” with “beliefs”. The form of argument wouldn’t change, except that you now say (absurdly) that beliefs, like preferences, can’t be irrational. I disagree with both.

One example of irrational desires is Akrasia (weakness of will). This phenomenon occurs when you want something (eat unhealthy, procrastinate, etc) but do not want to want it. In this case the former desire is clearly instrumentally irrational. This is a frequent and often serious problem and adequately labeled “irrational”.

Note that this is perfectly compatible with the brain having different parts: E.g.: The (rather stupid) cerebellum wants to procrastinate, the (smart) cortex wants to not procrastinate. When in contradiction, you should listen to your cortex rather than to your cerebellum. Or something like that. (Freud called the stupid part of the motivation system the “id” and the smart part the “super ego”.)

Such irrational desires are not reducible to actions. An action can fail to obtain for many reasons (perhaps it presupposed false beliefs) but that doesn’t mean the underlying desire wasn’t irrational.

Wants are not beliefs. They are things you feel.

Feelings and desires/”wants” are not the same. It’s the difference between hedonic and preference utilitarianism. Desires are actually more similar to beliefs, as both are necessarily about something (the thing which we believe or desire), whereas feelings can often just be had, without them being about anything. E.g. you can simply feel happy without being happy about something specific. (Philosophers call mental states that are about something “intentional states” or “propositional attitudes”.)

Moreover, sets of desires, just like sets of beliefs, can be irrational (“inconsistent”). For example, if you want x to be true and also want not-x to be true. That’s irrational, just like believing x while also believing not-x. A more complex example from utility theory: If $P$ describes your degrees of belief in various propositions, and $U$ describes your degrees of desire that various proposition are true, and $P (A and B) = 0$ , then $P (A or B) U (A or B) = P (A) U (A) + P (B) U (B)$ . In other words, if you believe two propositions to be mutually exclusive, your expected desire for their disjunction should equal the sum of your expected desires for the individual propositions, a form of weighted average.

cubefox 4 Dec 2024 22:34 UTC
2 points
0
in reply to: Alex_Altair’s comment on: The 2023 LessWrong Review: The Basic Ask
More specifically, for a Jeffrey utility function $U$ defined over a Boolean algebra of propositions, and some propositions $a, b$ , “the sum is greater than its parts” would be expressed as the condition $U (a \land b) > U (a) + U (B)$ (which is, of course, not a theorem). The respective general theorem only states that $U (a \land b) = U (a) + U (b ∣ a)$ , which follows from the definition of conditional utility $U (b ∣ a) = U (a \land b) - U (a) .$

cubefox 29 Nov 2024 17:08 UTC
2 points
0
in reply to: Sinclair Chen’s comment on: Sinclair Chen’s Shortform
Yeah definitional. I think “I should do x” means about the same as “It’s ethical to do x”. In the latter the indexical “I” has disappeared, indicating that it’s a global statement, not a local one, objective rather than subjective. But “I care about doing x” is local/subjective because it doesn’t contain words like “should”, “ethical”, or “moral patienthood”.

cubefox 26 Nov 2024 11:30 UTC
5 points
1
in reply to: Sinclair Chen’s comment on: Sinclair Chen’s Shortform
Ethics is a global concept, not many local ones. That I care more about myself than about people far away from me doesn’t mean that this makes an ethical difference.

cubefox 22 Nov 2024 20:24 UTC
3 points
0
on: I Have A New Paper Out Arguing Against The Asymmetry And For The Existence of Happy People Being Very Good
This seems to just repeat the repugnant conclusion paradox in more graphic detail. Any paradox is such that one can make highly compelling arguments for either side. That’s why it’s called a paradox. But doing this won’t solve the problem. A quote from Robert Nozick:

Given two such compelling opposing arguments, it will not do to rest content with one’s belief that one knows what to do. Nor will it do to just repeat one of the arguments, loudly and slowly. One must also disarm the opposing argument; explain away its force while showing it due respect.

cubefox 18 Nov 2024 21:28 UTC
6 points
2
in reply to: Leon Lang’s comment on: Leon Lang’s Shortform
Tailcalled talked about this two years ago. A model which predicts text does a form of imitation learning. So it is bounded by the text it imitates, and by the intelligence of humans who have written the text. Models which predict future sensory inputs (called “predictive coding” in neuroscience, or “the dark matter of intelligence” by LeCun) don’t have such a limitation, as they predict reality more directly.

cubefox 17 Nov 2024 16:56 UTC
2 points
0
in reply to: dirk’s comment on: Trying Bluesky
This still included other algorithmically determined tweets—from what your followers had liked and later more generally “recommended” tweets. These are no longer present in the following tab.

cubefox 17 Nov 2024 15:56 UTC
2 points
0
in reply to: dirk’s comment on: Trying Bluesky
I’m pretty sure there were no tabs at all before the acquisition.

cubefox 17 Nov 2024 14:59 UTC
2 points
0
in reply to: dirk’s comment on: Trying Bluesky
Twitter did use an algorithmic timeline before (e.g. tweets you might be interested in, tweets people you followed liked), it was just less algorithmic than the “for you” tab currently. The time when it was completely like the current “following” tab was many years ago.

cubefox 17 Nov 2024 3:42 UTC
3 points
0
in reply to: mako yass’s comment on: Trying Bluesky

The algorithm has been horrific for a while

After Musk took over, they implemented a mode which doesn’t use an algorithm on the timeline at all. It’s the “following” tab.

cubefox 17 Nov 2024 0:50 UTC
3 points
−2
in reply to: Maelstrom’s comment on: Alexander Gietelink Oldenziel’s Shortform
In the past we already had examples (“logical AI”, “Bayesian AI”) where galaxy-brained mathematical approaches lost out against less theory-based software engineering.

cubefox 15 Nov 2024 14:46 UTC
3 points
0
on: Seven lessons I didn’t learn from election day

Cities are very heavily Democratic, while rural areas are only moderately Republican.

I think this isn’t compatible with both getting about equally many votes. Because much more US Americans live in cities than in rural areas:

In 2020, about 82.66 percent of the total population in the United States lived in cities and urban areas.

https://www.statista.com/statistics/269967/urbanization-in-the-united-states/

cubefox 15 Nov 2024 13:58 UTC
6 points
2
in reply to: Leon Lang’s comment on: Leon Lang’s Shortform
It’s not that “they” should be more precise, but that “we” would like to have more precise information.

We know pretty conclusively now from The Information and Bloomberg that for OpenAI, Google and Anthropic, new frontier base LLMs have yielded disappointing performance gains. The question is which of your possibilities did cause this.

They do mention that the availability of high quality training data (text) is an issue, which suggests it’s probably not your first bullet point.

cubefox 14 Nov 2024 18:09 UTC
2 points
0
in reply to: Dalcy’s comment on: Darcy’s Shortform
Ah yes, the fork asymmetry. I think Pearl believes that correlations reduce to causations, so this is probably why he wouldn’t particularly try to, conversely, reduce causal structure to a set of (in)dependencies. I’m not sure whether the latter reduction is ultimately possible in the universe. Are the correlations present in the universe, e.g. defined via the Albert/Loewer Mentaculus probability distribution, sufficient to recover the familiar causal structure of the universe?