Beliefs? World model?
cubefox
Karma: 1,781
One example: For decades, we ate large amounts of vegetable shortening and margarine made from a chemical process that creates trans fats. Only relatively recently we became sure that trans fats cause heart disease, and restrictions about trans fat contents where put in place in many countries. See this Wikipedia article.
A major problem seems to be that many engineered things we eat every day were invented before rigorous food testing was mandatory. Since retroactively banning all this stuff is not realistic, we have to live with the risk of finding more such cases in the future. Similar things hold for other products, like specialized substances in food packaging, etc.
There is a strong correlation between someone boycotting a person for saying X and X being outside the Overton window. So a causal link is likely. People rarely boycott people for expressing things they disagree with but which are inside the Overton window.
It is also possible that Bob is racist in the sense of successfully working to cause unjust ethnic conflict of some kind, but also Bob only says true things. Bob could selectively emphasize some true propositions and deemphasize others.
Sure, though this is equally possible for the opposite: When Alice is shunning or shaming or cancelling people for expressing or defending a taboo hypothesis, without her explicitly arguing that the hypothesis is false or disfavored by the evidence. In fact, this is usually much easier to do than the former, since defending a taboo hypothesis is attached to a large amount of social and career risk, while attacking a taboo hypothesis is virtually risk-free. Moreover, attacking a taboo hypothesis will likely cause you to get points from virtue signalling.
I think the following resembles a motte-and-bailey pattern: Bailey: “He is a racist, people may want to explain why racism is terrible.” Motte: “Oh I just meant he argued for the empirical proposition that there are heritable statistical group differences in IQ.” Accusing someone of racism is a massively different matter from saying that he believes there are heritable group differences in IQ. You can check whether a term is value neutral by whether the accused people apply it to themselves, in this case they clearly do not. The term “racist” usually carries the implication or implicature of an attitude that is merely based on an irrational prejudice, not an empirical hypothesis with reference to a significant amount of statistical and other evidence.
I find already labelling someone who holds an empirical proposition (which is true or false) as “racist” (which is a highly derogatory term, not a value neutral label) is defamatory. The vague hinting about alleged “rumours” here also seems to just serve to make him appear in a bad light.
I find this attitude sad. I think his blog is currently clearly one of the best ones on the Internet. Even if you don’t agree with some of his positions, I take it to be a deeply anti-rational attitude to try to shun or shame people for saying things that are outside the Overton window. Especially when he has clearly proven on his website that he has highly nuanced takes on various other, less controversial, topics. It reminds me of people trying to shame Scott Alexander for daring to step a little outside the Overton window himself.
In my opinion, true rationalists should exactly not react to such takes with “he said something taboo, let’s boycott things where he is involved”. If you disagree with him, a better attitude would be to write a post about one of the articles on his website, concretely indicating and rebutting things where you think he is wrong. Only claiming “I do not think Cremieux meets this standard of care and compassion” is so vague of an accusation that I don’t know whether you even disagree with anything he said. It sounds like low decoupling and tone policing. I wrote more on rationalist discourse involving taboos here.
It seems highly likely that the majority of humans prefer humanity not to be replaced by AI in the foreseeable future. So I don’t think there is that much variance here.
In a preference utilitarian calculus it does matter which possible preferences actually exist and which don’t. We don’t count hypothetical votes in an election either.
But from a consistent description doesn’t follow concrete existence. Otherwise an ontological argument could imply the existence of God. And if someone doesn’t exist, their parts don’t exist either. A preference is a part of someone, like an arm or a leg, and they wouldn’t exist. A description could only answer what preference someone would have if they existed. But that’s only a hypothetical preference, not one that exists.
You can think of what is good for someone as preferences. And you can think of preferences as being about potentially switching from some state X (where you currently are) to Y. There can be no preferences about switching from X to Y if you don’t exist in X, since there is nobody who would have such a preference. Of course you can have preferences once you exist in Y, e.g. about not switching back to X (not dying). But that only means that switching from Y to X is bad, not that switching from X to Y was good.
Well, like any preference it could change over time, but “final” is the opposite of “instrumental”, not of “variable”.
You could use the same argument about existing beings against longtermism, but I just don’t think it carves reality at its joints. Your responsibility toward possible beings is no different than your responsibility toward existant beings. You could make things better or worse for either of them and they’d love or hate that if you did.
The important point is: If we don’t let them come into existence then they would not hate that. Merely possible beings don’t exist and therefore have no preference for coming to be. So it isn’t bad if they never get to exist. See here for details.
If Galaxy-brain Gavin’s theory has serious counterintuitive results, like ignoring the preference of current humans that humanity be not replaced by AI in the future, then Gavin’s theory is not galaxy-brained enough.
Instead, my opposition to AI successionism comes from a preference toward my own kind. This is hardwired in me from biology. I prefer my family members to randomly-sampled people with similar traits. I would certainly not elect to sterilize or kill my family members so that they could be replaced with smarter, kinder, happier people.
I would go further. Most people probably also prefer humans existing in the future to AIs existing in the future. It’s not just about direct personal descendants, but about a preference for our own species continuing to exist.
Is that preference guilty of “speciesism” and therefore immoral? No, it’s just a final preference about what should happen in the future. It’s not immoral, since possible future AIs do not exist yet and as such have no preferences (like coming into existence) which could be violated. So it isn’t bad when we don’t make them exist. This is called the procreation asymmetry.
I agree with this. Just one quibble here:
Utilitarianism implies that if we build an AI that successfully maximizes utility/value, we should be ok with it replacing us.
That is true for some forms of utilitarianism (total or average utilitarianism) but not necessarily for some others (person affecting preference utilitarianism). For the reason you outline, I think the latter kind of utilitarianism is correct. Its notion of utility/value incorporates what currently existing people want, and I believe that most humans want that humanity will not be replaced by AI in the future.
One way to think about it: AI succesionism is bad for a similar reason we regard being dead as bad: We don’t want it to happen. We don’t want to die, and we also don’t want humanity to become extinct through sterilization or other means. Perhaps we have these preferences for biological reasons, but it doesn’t matter why: final goals need no justification, they just exist and have moral weight in virtue of existing. The fact that we wouldn’t be around if it (being dead, or humanity being extinct) happens, and that we therefore would not suffer from it, doesn’t invalidate the current weight of our current preferences.
Though pedantically I wouldn’t exactly call this Bayesian, as already Descartes said similar things.
One reason for this (see here) may be that RLVR models prefer giving an answer with very low probability of being correct to answering that they don’t know. (Though that doesn’t explain their inability to properly see the Pokémon game.)
Maybe at some point you want to write a post on why you think high agency could or would not be automated.
As far as I can understand it, this doesn’t seem to contain an argument for why you think highly general AIs aren’t possible.
I think more posts should be formatted as a nested list. They are especially clear, since indentation ≈ elaboration, which is not visible in continuous text.