AIS student, self-proclaimed aspiring rationalist, very fond of game theory.
”The only good description is a self-referential description, just like this one.”
momom2
Karma: 204
distraction had no effect on identifying true propositions (55% success for uninterrupted presentations, vs. 58% when interrupted); but did affect identifying false propositions (55% success when uninterrupted, vs. 35% when interrupted)
If you are confused by these numbers (why so close to 50%? Why below 50%) it’s because participants could pick four options (corresponding to true, false, don’t know and never seen).
You can read the study, search for keyword “The Identification Test”.
I don’t see what you mean by the grandfather problem.
I don’t care about the specifics of who spawns the far future generation; whether it’s Alice or Bob I am only considering numbers here.
Saving lives now has consequences for the far future insofar as current people are irrepleceable: if they die, no one will make more children to compensate, resulting in a lower total far future population. Some deaths are less impactful than others for the far future.
That’s an interesting way to think about it, but I’m not convinced; killing half the population does not reduce the chance of survival of humanity by half.
In terms of individuals, only the last <.1% matter (not sure about the order of magnitude, but in any case it’s small as a proportion of the total).
It’s probably more useful to think in terms of events (nuclear war, misaligned ASI → prevent war, research alignment) or unsurvivable conditions (radiation, killer robots → build bunker, have kill switch) that can prevent humanity from recovering from a catastrophe.
Yes, that’s the first thing that was talked about in my group’s discussion on longtermism. For the sake of the argument, we were asked to assume that the waste processing/burial choice amounted to a trade in lives all things considered… but the fact that any realistic scenario resembling this thought experiment would not be framed like that is the central part of my first counterargument.
Two arguments against longtermist thought experiments
I enjoy reading any kind of cogent fiction on LW, but this one is a bit too undeveloped for my tastes. Perhaps be more explicit about what Myrkina sees in the discussion which relates to our world?
You don’t have to always spell earth-shattering revelations out loud (in fact it’s best to let the readers reach the correct conclusion by themselves imo), but there needs to be enough narrative tension to make the conclusion inevitable; as it stands, it feels like I can just meh my way out of thinking more than 30s on what the revelation might be, the same way Tralith does.
Thanks, it does clarify, both on separating the instantiation of an empathy mechanism in the human brain vs in AI and on considering instantiation separately from the (evolutionary or training) process that leads to it.
I was under the impression that empathy explained by evolutionary psychology as a result of the need to cooperate with the fact that we already had all the apparatus to simulate other people (like Jan Kulveit’s first proposition).
(This does not translate to machine empathy as far as I can tell.)
I notice that this impression is justified by basically nothing besides “everything is evolutionary psychology”. Seeing that other people’s intuitions about the topic are completely different is humbling; I guess emotions are not obvious.
So, I would appreciate if you could point out where the literature stands on the position you argue against, Jan Kulveit’s or mine (or possibly something else).
Are all these takes just, like, our opinion, man, or is there strong supportive evidence for a comprehensive theory of empathy (or is there evidence for multiple competing theories)?
I do not find this post reassuring about your approach.
Your plan is unsound; instead of a succession of events which need to go your way, I think you should aim for incremental marginal gains. There is no cost-effectiveness analysis, and the implicit theory of change is lacunar.
Your press release is unreadable (poor formatting), and sounds like a conspiracy theory (catchy punchlines, ALL CAPS DEMANDS, alarmist vocabulary and unsubstantiated claims) ; I think it’s likely to discredit safety movements and raise attention in counterproductive ways.
The figures you quote are false (the median from AI Impacts is 5%) or knowingly misleading (the numbers from Existential risk from AI survey are far from robust and as you note, suffer from selection bias), so I think it’s fair to call them lies.
Your explanations for what you say in the press release sometimes don’t make sense! You conflate AGI and self-modifying systems, your explanation for “eventually” does not match the sentence.
Your arguments are based on wrong premises—it’s easy to check that your facts such as “they are not following the scientific method” are plain wrong. It sounds like you’re trying to smear OpenAI and Sam Altman as much as possible without consideration for whether what you’re saying is true.
I am appalled to see this was not downvoted into oblivion! My best guess is that people feel that there are not enough efforts going towards stopping AI and did not read the post and the press release to check that you have good reason motivating your actions.
I agree with the broad idea, but I’m going to need a better implementation.
In particular, the 5 criteria you give are insufficient because the example you give scores well on them, and is still atrocious: if we decreed that “black people” was unacceptable and should be replaced by “black peoples”, it would cause a lot of confusion on account of how similar the two terms are and how ineffective the change is.
The cascade happens because of a specific reason, and the change aims at resolving that reason. For example, “Jap” is used as a slur, and not saying it shows you don’t mean to use a slur. For black people/s, I guess the reason would be something like not implying that there is a single black people, which only makes sense in the context of a specialized discussion.
I can’t adhere to the criteria you proposed because they don’t work, and I don’t want to bother thinking that deep about every change of term on an everyday basis, so I’ll keep on using intuition to choose when to solve respectability cascades for now.
For deciding when to trigger a respectability cascade, your criteria are interesting for having any sort of principled approach, but I’m still not sure they outperform unconstrained discussion on the subject (which I assume is the default alternative for anyone who cares enough about deliberately triggering respectability cascades to have read your post in the first place).
A lot of your AI-risk reason to support Harris seems to hinge on this, which I find very shaky. How wide are your confidence intervals here?
My own guesses are much more fuzzy. According to your argument, if my intuition was .2 vs .5, then it’s an overwhelming case for Harris but I’m unfamiliar enough with the topic that it could easily be the reverse.
I would greatly appreciate more details on how you reach your numbers (and if they’re vibes, reason whether to trust those vibes).
Alternatively, I feel like I should somehow discount the strength of the AI-risk reason based on how likely I think these numbers are to more or less hold true, but I don’t know a principled way to do it.
Piling bounded arguments
Seems like you need to go beyond arguments of authority and stating your conclusions and instead go down to the object-level disagreements. You could say instead “Your argument for ~X is invalid because blah blah” and if Jacob says “Your argument for the invalidity of my argument for ~X is invalid because blah blah” then it’s better than before because it’s easier to evaluate argument validity than ground truth.
(And if that process continues ad infinitam, consider that someone who cannot evaluate the validity of the simplest arguments is not worth arguing with.)
It’s thought-provoking.
Many people here identify as Bayesians, but are as confused as Saundra by the troll’s questions, which indicates that they’re missing something important.
It wasn’t mine. I did grow up in a religious family, but becoming a rationalist came gradually, without sharp divide with my social network. I always figured people around me were making all sorts of logical mistakes though, and noticed very early deep flaws in what I was taught.
It’s not. The paper is hype, the authors don’t actually show that this could replace MLPs.
This is very interesting!
I did not expect that Chinese would be more optimistic about benefits than worried about risks and that they would rank it so low as an existential risk.
This is in contrast with posts I see on social media and articles showcasing safety institutes and discussing doomer opinions, which gave me the impression that Chinese academia was generally more concerned about AI risk and especially existential risk than the US.
I’m not sure how to reconcile this survey’s results with my previous model. Was I just wrong and updating too much on anecdotal evidence?
How representative of policymakers and of influential scientists do you think these results are?
About the Christians around me: it is not explicitly considered rude, but it is a signal that you want to challenge their worldview, and if you are going to predictably ask that kind of question often, you won’t be welcome in open discussions.
(You could do it once or twice for anecdotal evidence, but if you actually want to know whether many Christians believe in a literal snake, you’ll have to do a survey.)
I disagree – I think that no such perturbations exist in general, rather than that we have simply not had any luck finding them.
I have seen one such perturbation. It was two images of two people, one which was clearly male and the other female, though I wasn’t be able to tell any significant difference between the two images on 15s of trying to find one except for a slight difference in hue.
Unfortunately, I can’t find this example again on a 10mn search. It was shared on Discord; the people in the image were white and freckled. I’ll save it if I find it again.
The pyramids and Mexico and the pyramids in Egypt are related via architectural constraints and human psychology.
Top of the head like when I’m trying to frown too hard