Not going to respond to all these, a lot seem like nitpicks.
Your original claim included the phrase “the only possible alternative hypothesis”, so this seems totally non-responsive to my problem with it.
My other point was that the problems were still experienced “at MIRI” even if they were caused by other things in the social environment.
That’s good to know, thanks. I think it would make your point here much stronger and more legible if those specific details were included in the original claim.
Edited.
Compare: “MIRI already knew about a working AGI design and that it would not be that hard for me to come up with a working AGI design on short notice just by thinking about it” with “the pieces to make AGI are likely already out there and someone just has to put them together… he encouraged me to figure it out for myself, saying it was within my abilities to do so”. The first is making several much stronger claims than the second (w.r.t. certainty & specific knowledge of a working design, and their belief in how difficult it would be for you to generate one yourself under what timeframe), in a way that makes MIRI/Nate seem much more unreasonable.
Nate implied he had already completed the assignment he was giving me.
The assignment wouldn’t provide evidence about whether the pieces to make AGI are already out there unless it was “workable” in the sense that iterative improvement with more compute and theory-light technique iteration would produce AGI.
If you agree that Scott & Michael have critical disagreements about the trade-offs in a way that’s relevant to the question at hand—more so than the surface-level agreement Scott’s writing demonstrates—why is this included at all?
Edited to make it clear that they disagree. The agreement is relevant to place a bound on the scope of what they actually disagree on.
I’m not following the chain of logic here. Ziz claims that Anna told her she thought Ziz was likely to be net negative (in the context of AI safety research), after Ziz directly asked her if she thought that. Are you claiming that Anna was sufficiently familiar with the details of Ziz’s ontology (which, afaik, she hadn’t even developed in any detail at that point?) to predict that it might tempt Ziz to commit suicide?
She might have guessed based on Ziz’s utilitarian futurism (this wouldn’t require knowing many specific details), or might not have been thinking about that consciously. It’s more likely she was trying to control Ziz (she has admitted to generally controlling people around CFAR by e.g. hoarding info). I think my general point is that people are trying to memetically compete with each other in ways that involve labeling others “net negative” in a way that people can very understandably internalize and which would lead to suicide. It’s more like a competition to drive each other insane than one to directly kill each other. A lot of competition (e.g. the kind that would be predicted by evolutionary theory) is subconscious and doesn’t indicate legal responsibility.
Anyway, I edited to make it clearer that many of the influences in question are subconscious and/or memetic.
I predict that if we anonymously polled MIRI researchers (or AI alignment researchers more broadly), very few of them would endorse “thinking about extreme AI torture scenarios [is] part of my job”, if it carries the implication that they also need to think about those scenarios in explicit detail rather than “many 0s, now put a minus sign in front of them”.
I predict that they would say that having some philosophical thoughts about negative utilitarianism and related considerations would be part of their job, and that AI torture scenarios are relevant to that, although perhaps not something they would specifically need to think about.
So it sounds like you agree that to the extent that it was part of your job “to imagine myself in the role of someone who is going to be creating the AI that could make everything literally the worst it could possibly be”, that was an inference you drew (maybe reasonably!) from the environment, rather than someone at MIRI telling you explicitly that it was part of your job (or strongly implying the same)?
Edited to make this clearer.
The original claim you were ridiculing was not that “AI is likely to come soon”.
They’re highly related, having a working AGI design is an argument for short timelines.
He is providing a reason why he himself does not make much of a habit of discussing AI timelines, and that reason is that he is worried about the mental health of others, not that he thinks discussing it pessimizes for timeline outcomes.
Sure, I mentioned it as a consideration other than the consideration I already mentioned about making AI come sooner.
I think the fact that Michael Vassar directly interacted with you during the relevant timeframe in a way which you yourself think made things worse is notable, in the sense that for most plausible sets of priors, you should probably be updating upwards on the hypothesis that “spending time around Michael Vassar is more likely lead to psychosis than spending time around most other people”, irrespective of his state of knowledge & motivations at the time.
I agree it’s weak evidence for that proposition. However the fact that he gave me useful philosophical advice is evidence against that proposition. In total the public info Scott and I have revealed provides very little directional evidence about this proposition.
Not going to respond to all these, a lot seem like nitpicks.
My other point was that the problems were still experienced “at MIRI” even if they were caused by other things in the social environment.
Edited.
Nate implied he had already completed the assignment he was giving me.
The assignment wouldn’t provide evidence about whether the pieces to make AGI are already out there unless it was “workable” in the sense that iterative improvement with more compute and theory-light technique iteration would produce AGI.
Edited to make it clear that they disagree. The agreement is relevant to place a bound on the scope of what they actually disagree on.
She might have guessed based on Ziz’s utilitarian futurism (this wouldn’t require knowing many specific details), or might not have been thinking about that consciously. It’s more likely she was trying to control Ziz (she has admitted to generally controlling people around CFAR by e.g. hoarding info). I think my general point is that people are trying to memetically compete with each other in ways that involve labeling others “net negative” in a way that people can very understandably internalize and which would lead to suicide. It’s more like a competition to drive each other insane than one to directly kill each other. A lot of competition (e.g. the kind that would be predicted by evolutionary theory) is subconscious and doesn’t indicate legal responsibility.
Anyway, I edited to make it clearer that many of the influences in question are subconscious and/or memetic.
I predict that they would say that having some philosophical thoughts about negative utilitarianism and related considerations would be part of their job, and that AI torture scenarios are relevant to that, although perhaps not something they would specifically need to think about.
Edited to make this clearer.
They’re highly related, having a working AGI design is an argument for short timelines.
Sure, I mentioned it as a consideration other than the consideration I already mentioned about making AI come sooner.
I agree it’s weak evidence for that proposition. However the fact that he gave me useful philosophical advice is evidence against that proposition. In total the public info Scott and I have revealed provides very little directional evidence about this proposition.