AnthonyC

Karma: 3,014

AnthonyC May 16, 2025, 8:17 PM
3 points
0
on: reflecting on criticism
It sounds like, on reflection, your previous post was less about reduction, and more about misapplying the idea of reduction in a way that ignores or elides map-territory distinctions, instead pretending our best known current map is actually reality. Would you agree with that?

AnthonyC May 12, 2025, 12:09 PM
4 points
0
in reply to: Viliam’s comment on: [Book Translation] Three Days in Dwarfland
Yes, my thinking is similar. Elementary school teachers often barely understand the math they are required to be teaching, and don’t have the fluidity needed to handle a more free-flowing discussion about a book that doesn’t conform to a specific curriculum. The whole system frequently retreats into drilling specific procedures that mean nothing to the teachers and students involved, even when the explicit stated goal is to help build understanding and problem solving skills. The idea that math classes even could include reading books is just not part of the conversation. Only English classes assign books to read—not history, not foreign languages, and definitely not science and math. Related: I had exactly one math teacher, in seventh grade, who assigned a term paper on any math topic of our choice. I got a 70, the lowest math grade I ever received in any year, and it was because, as he told me in his own words, he didn’t understand what I’d written and couldn’t follow it.
I will say, there are some English language books that deliberately incorporate math in ways that are both fun and educational, if you had a teacher able and willing to lead such discussions. There’s many such books by Ian Stewart. Alice in Wonderland would be a fair choice, and the kids probably already know the story. For middle or high schoolers especially, it doesn’t have to just be fiction, either. For the “When will we ever need this?” crowd, something like Nonplussed or Impossible?, both by Julian Haveil, could be a welcome and eye-opening change of pace.

AnthonyC May 12, 2025, 2:01 AM
4 points
0
on: [Book Translation] Three Days in Dwarfland
I love things like this, and always wondered why we never had these kinds of books as part of math curricula in elementary and middle school in the US.

AnthonyC May 12, 2025, 1:42 AM
2 points
0
in reply to: TFD’s comment on: Where is the YIMBY movement for healthcare?
Yes, true, but somehow we don’t have that problem with veterinary care, even when there’s insurance involved. I don’t really know how likely it is for any given treatment to help my cat, or for how long, but the vet gives me a list of options and each of their prices, in advance, and then that’s what I pay. I pick based on a combination of my understanding, their recommendations, and my budget. It’s generally far more humane, more empowering, and less condescending than getting care for a human, because our society lets people take responsibility for their pets in ways it doesn’t let adults do for themselves.
Even besides that, though, the reality is much, much worse than that in (human) medicine.
Depending on whether I have insurance and exactly which kind, the base price of a service—not what I ultimately pay, but the total number that I and my insurer pay—can vary by more than an order of magnitude. Even after the fact it can be really difficult to know how much anyone is paying anyone else. I’ve had three different situations, with different providers and insurers, in which the provider kept applying payments to the wrong line items, in ways that messed up who was supposed to pay what and when, that took months of calendar time and tens of hours of time on the phone to sort out.
As you noted, goods like prescriptions should be simpler than medical services to price out. But, when I have to fill the same prescription in different pharmacies (which is every month, because I travel full time), the price has varied by as much as a factor of a hundred between pharmacies or even by 2-10x month to month from the same pharmacy. The price depends on the insurer. The price can sometimes be higher with insurance than without, because I can’t combine insurance with various magic-seeming discount programs like GoodRx that are available to anyone and that some pharmacists will apply for you without you even asking. But it can sometimes also be ultimately cheaper to pay the higher price anyway depending on how your deductibles and copays work and when the magic plan year end date happens. Many pharmacies won’t tell you the price before your Rx is in their system, and once it is in the system, you may not be able to change whether or not to use insurance. For many medications it is difficult or illegal to move them to a different pharmacy at all, or it can only be done a certain number of times, or it can only be done after the first pharmacy has filled it at least once.

AnthonyC May 9, 2025, 3:01 AM
5 points
2
on: a brief critique of reduction
I can’t tell if this is intended to be taken seriously or not, and I won’t bother pointing out the various individual false assumptions, misunderstandings, reasoning errors, non sequiturs, or contentless statements. Any modern LLM can handle that just fine if you want to know. But this sentence caught my eye:
Reduction is an operation of reason by the observer to extract the most relevant relations from the observed.
This is a misunderstanding of what “reduction” actually means, but I think it’s a very common one. I can totally see how, if that’s what you believe the word means, you would come to believe many of the other claims in this post. What this describes, though, is a form of fake reduction, and I really do recommend you take the time to read the Reductionism 101 sequence, especially the last 4 posts in it. Real reduction requires quite a bit more knowledge and understanding and perspective than most people imagine. See also the first handful of posts from Joy in the Merely Real.

AnthonyC May 8, 2025, 3:37 AM
2 points
−1
on: OpenAI Claims Nonprofit Will Retain Nominal Control
One thing that’s not clear to me (and you may have discussed this in the previous posts, I don’t remember) is: was the previous structure even legally valid/enforceable? Can you write into the structure of a for-profit LLC that it has to act in accordance with some goal other than profit? Because as I understand it, a board member has a fiduciary duty to the company regardless of their own interests, or that of the organization or process that made them a board member. Someone recently highlighted to me some examples of cases (in normal for-profit startups) where this gives you behavior like board members approving some measure, and then the same individuals, now acting as shareholders where they can do as they please, vote against it.
Maybe the original OpenAI structure included a clever and enforceable way around this. But if not, then maybe it’s possible the switch to a PBC closes a loophole whereby investors could have sued the board for acting according to the nonprofit’s interests instead of their own.

AnthonyC May 8, 2025, 12:58 AM
4 points
0
on: Social status games might have “compute weight class” in the future
My instinctive response is: weight classes are for controlled competitions where fairness is what we actually want. For social status games, if you want to enforce weight classes, you need a governing body who gets to define the classes and define the rules of the game, but the rules of social status games frequently include being not fully expressible in precise terms. This isn’t necessarily a showstopper, but it necessarily includes admitting what range of the hierarchy you’re in and cannot rise above. As I understand it, the reason the self-sorting works today is that when people compete in the wrong weight classes, it’s not fun for either side. A Jupiter Brain might theoretically be amenable to playing a social game with me on my level, but at best it would be like me playing tic-tac-toe with a little kid, where the kid is old enough to realize I’m throwing the game but not old enough to have solved the game.
Personally I’d much rather not spend my time on such games when it is possible to manage that. But I don’t always have that choice now, and probably still won’t at least sometimes in the future.

AnthonyC May 8, 2025, 12:30 AM
3 points
0
in reply to: Stephen Martin’s comment on: Utah Court Case Over State Law Regarding “Personhood” for Nonhuman Intelligences
Thanks! This is an interesting angle I wasn’t much thinking about.

AnthonyC May 6, 2025, 3:58 PM
3 points
0
on: Utah Court Case Over State Law Regarding “Personhood” for Nonhuman Intelligences
I anticipate this will lead to some interesting phrasing choices around the multiple meanings of “conception” as the discussions on what and how and whether AI’s ‘really’ think continue to evolve.

AnthonyC May 5, 2025, 9:19 PM
2 points
0
on: Tsinghua paper: Does RL Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?
There’s a story about trained dolphins. The trainer gave them fish for doing tricks, which worked great. Then they decided to only give them fish for novel tricks. The dolphins, trained under the old method, ran through all the tricks they knew, got frustrated for a while, then displayed a whole bunch of new tricks all at once.
Among animals, RL can teach specific skills but also reduces creativity in novel contexts. You can train creative problem solving, but in most cases, when you want control of outcomes, that’s not what you do. The training for creativity is harder, and less predictable, and requires more understanding and effort from the trainer.
Among humans, there is often a level where the more capable find supposedly simple questions harder, often because they can see all the places where the question assumes a framework that is not quite as ironclad as the asker thinks. Sometimes this is useful. More often it is a pain for both parties. Frequently the result is that the answerer learns to suppress their intelligence instead of using it.
In other words—this post seems likely to be about what this not-an-AI-expert should expect to happen.

AnthonyC May 5, 2025, 8:51 PM
2 points
0
on: AI #114: Liars, Sycophants and Cheaters
He makes some bizarre statements, such as that if you have a rare gene that might protect you from the AI having enough data to have ‘a good read’ on you, and that genetic variation will ‘protect you from high predictability.’
You know, even if this were true… if you’re a less unpredictable entity in a world where sufficiently power AI wants to increase predictability, there are many simple and obvious classes of interventions that reliably achieve that. Mostly, those interventions look nothing like freedom, and you’re not going to like them.

AnthonyC May 5, 2025, 4:23 PM
15 points
6
on: Legal Supervision of Frontier AI Labs is the answer.
I’m sympathetic to the point of view that this is necessary, though I wouldn’t call it “the” answer—I don’t think we can have high enough confidence that it is sufficient. That said, while you mention the reasons for skepticism of applying existing legal frameworks (which I agree with!), I think the hard step is writing the proposed new rules down.
What does a clear legal mandate look like? What are the requirements, which we are capable of writing down with legally-enforceable precision, that would (or at least could) be adequate without being predictably thrown out by courts or delayed so long they end up not mattering? How many people exist who are capable of making the necessary evaluations, and is the government capable of hiring enough of them?

AnthonyC May 5, 2025, 1:34 AM
12 points
10
in reply to: Nina Panickssery’s comment on: Why I am not a successionist
There’s no reason for me to think that my personal preferences (e.g. that my descendants exist) are related to the “right thing to do”, and so there’s no reason for me to think that optimizing the world for the “right things” will fulfil my preference.
This, and several of the passages in your original post such as, “I agree such a definition of moral value would be hard to justify,” seem to imply some assumption of moral realism that I sometimes encounter as well, but have never really found convincing arguments for. I would say that the successionists you’re talking to are making a category error, and I would not much trust their understanding of ‘should’-ness outside normal day-to-day contexts.
In other words: it sounds like you don’t want to be replaced under any conditions you can foresee.. You have judged. What else is there?
I can’t really imagine a scenario where I “should” or would be ok with currently-existing-humans going extinct, though that doesn’t mean none could exist. I can, however, imagine a future where humanity chooses to cease (most?) natural biological reproduction in favor of other methods of bringing new life into the world, whether biological or artificial, which I could endorse (especially if we become biologically or otherwise immortal as individuals). I can further imagine being ok with those remaining biological humans each changing (gradually or suddenly) various aspects of their bodies, their minds, and the substrates their minds run on, until they are no longer meat-based and/or no longer ‘human’ in various ways most people currently understand the term.

AnthonyC May 3, 2025, 1:06 AM
2 points
−3
on: OpenAI Preparedness Framework 2.0
I know this can all be adequately explained by perfectly normal human motivations, but there’s still a small part of me that wonders if some of the unfortunate changes are being influenced by some of the very factors (persuasion, deception, sandbagging, etc.) that are potentially so worrying.

AnthonyC May 2, 2025, 2:39 PM
6 points
2
on: The Continuum Fallacy and its Relatives
Whenever something like this comes up, I have a standard reply that looks like this: “Ok, sure, neither of us can define a ‘salad,’ but would you agree that we can can define an ordinal scale of ‘saladness’ and mostly agree on the relative orderings of different things along it? Good, then I think we’re done here, all that’s left is the social convention about where along that axis we stop using the word.”
Also, I recently had (a self-aware and joking version of) this exact conversation where someone said ‘purple’ isn’t real. Again, simple answer: From a physics perspective, obviously there are no ‘real’ color categories. From a psychological perspective, obviously we use a three-axis color representation, where ‘purple’ is as well-defined as any other combination. The answer is entirely about why you’re asking the question.

AnthonyC Apr 29, 2025, 1:45 PM
5 points
0
in reply to: MattJ’s comment on: Our Reality: A Simulation Run by a Paperclip Maximizer
This does not imply that the simulation is run entirely in linear time, or at a constant frame rate (or equivalent), or that details are determined a priori instead of post hoc. It is plausible such a system could run a usually-convincing-enough simulation at lower fidelity, back-calculate details as needed, and modify memories to ignore what would have been inconsistencies when doing so is necessary or just more useful/tractable. ‘Full detail simulation at all times’ is not a prerequisite for never being able to find and notice a flaw, or for getting many kinds of adequate high level macroscopic outputs.
In other words: If I want to convince you something is a real tree, it needs to look and feel like a tree, but it doesn’t need an exact, well-defined wave-function. Classical approximations at tens of microns scale are about the limit of unaided human perception. If you pull out a magnifying glass or a scanning electron microscope, then you can fill in little pieces of the remaining whole, but you still aren’t probing the whole tree down to the Planck scale.

AnthonyC Apr 28, 2025, 3:23 PM
4 points
1
in reply to: YonatanK’s comment on: Three Months In, Evaluating Three Rationalist Cases for Trump
That’s true, and you’re right, the way I wrote my comment overstates the case. Every individual election is complicated, and there’s a lot more than one axis of variation differentiation candidates and voters. The whole process of Harris becoming the candidate made this particular election weird in a number of ways. And as a share of the electorate, there are many fewer swing voters than there used to be a few decades ago, and not conveniently sorted into large, coherent blocks.
And yet, it’s also true that as few as ~120,000 votes in WI, MI, and PA could have swung the result, three moderate states that have flipped back and forth across each of the past four presidential elections. Only slightly more for several other combinations of states. It’s not some deep mystery who lives in the rust belt, and what positions on issues a few tens of thousands of voters who are on the fence might care about. It’s not like those issues are uncorrelated, either. And if you look at the last handful of elections, a similar OOM of voters in a similar set of states could have swung things either way, each time.
And it’s true that Harris underperformed Biden-2024 by vote share in every state but Utah (and 37.7% vs 37.8% does not matter to the outcome in any plausible scenario). If I’m reading the numbers correctly she also received fewer votes numerically than Biden in all but 6 states.
So yes: I can very easily imagine scenarios where you’re right, and the fact that we don’t meet the theoretical assumptions necessary for the median voter theorem to apply means we can’t assume an approximation of it in practice. It’s even possible, if the Dems had really started organizing sustained and wide-ranging GOTV campaigns fifteen years ago, that there could be the kinds of blue wave elections I keep getting told are demographic inevitabilities just around the corner, as long as they keep moving further towards the current set of progressive policy goals. But what I cannot imagine is that, in July 2024, in the country as it actually existed, Harris wouldn’t have done better by prioritizing positions (many of which she actually already said she held!) that a relative handful of people in an actual handful of states consistently say they care the most about, and explaining why Trump’s usually-only-vaguely-gestured-at plans would make many of their situations worse. Would it have been enough? I don’t know. But it is a better plan than what happened, if what you want is to win elections in order to govern.

AnthonyC Apr 26, 2025, 2:02 PM
2 points
0
on: AI #113: The o3 Era Begins
Apparently the new ChatGPT model is obsessed with the immaculate conception of Mary
I mean, “shoggoth” is not that far off from biblically accurate angels… ;-)

AnthonyC Apr 26, 2025, 12:06 AM
2 points
0
in reply to: Eggs’s comment on: Fish and Faces
I’d say that in most contexts in normal human life, (3) is the thing that makes this less of an issue for (1) and (2). If the thing I’m hearing about it real, I’ll probably keep hearing about it, and from more sources. If I come across 100 new crazy-seeming ideas and decide to indulge them 1% of the time, and so do many other people, that’s usually, probably enough to amplify the ones that (seem to) pan out. By the time I hear about the thing from 2, 5, or 20 sources, I will start to suspect it’s worth thinking about at a higher level.

AnthonyC Apr 25, 2025, 11:55 PM
3 points
0
in reply to: ryan_greenblatt’s comment on: AI 2027 is a Bet Against Amdahl’s Law
Exactly. More fundamentally, that is not a probability graph, it’s a probability density graph, and we’re not shown the line beyond 2032 but just have to assume the integral from 2100-->infinity is >10% of the integral from 0-->infinity. Infinity is far enough away that the decay doesn’t even need to be all that slow for the total to be that high.