I appreciate that you took the time to respond to my post in detail. I explained at the top why I had a difficult time engaging productively with your post (i.e. learning from it). I did learn some specific things, such as the claimed sequence of events prior to Maia’s suicide, and Nate’s recent retraction of his earlier public statement on OpenAI. Those are things which are either unambiguously claims about reality, or have legible evidence supporting them.
I mentioned:
I did it too.
Ziz labeling it as an infohazard is in compliance with feedback Ziz has received from community leaders.
People didn’t draw attention to Fluttershy’s recent blog post, even after I posted about it.
None of these carry the same implication that the community, centrally, was engaging in the claimed concealment. This phrasing deflects agency away from the person performing the action, and on to community leaders: “Ziz labeling it as an infohazard is in compliance with feedback Ziz has received from community leaders.” “Ziz labeled something that might have contributed to Maia’s suicide an infohazard, possibly as a result of feedback she got from someone else in the community well before she shared that information with Maia” implies something very different from “Many others have worked to conceal the circumstances of their deaths”, which in context makes it sound like an active conspiracy engaged in by central figures in the community. People not drawing attention to Fluttershy’s post, even after you posted about it, is not active concealment.
> In theory, the problems you experienced could have come from sources other than your professional environment. That is a heck of a missing middle.
Most of the post is an argument that the professional environment contributed quite a lot.
Your original claim included the phrase “the only possible alternative hypothesis”, so this seems totally non-responsive to my problem with it.
> I don’t know what Michael’s views on the subject actually are, but on priors I’m extremely skeptical that the correspondence is sufficient to make this a useful comparison (which, as an appeal to authority, is already on moderately shaky grounds).
Obviously he’s going to disagree with them on specifics, I’m mentioning them as agreeing on the general view Scott attributed to Michael.
Again, this seems non-responsive to what I’m saying is the issue, which is that the “general view” is more or less useless for evaluating how much in “agreement” they really are, as opposed to the specific details.
Not an employee. Did some teaching at CFAR events. Implied they were telling me information about a CFAR employee’s opinion. Even if they’re repeating a rumor, that still implies use of that psychedelic is common in the social circle, even if that doesn’t mean CFAR caused it at all.
That’s good to know, thanks. I think it would make your point here much stronger and more legible if those specific details were included in the original claim.
The distinction I am making is about (a) treating DeepMind as a participant in discourse, who can be convinced by reasoned argument to do better things, or (b) treating DeepMind as a competitor in a race, who can’t be reasoned with, but has to be beaten decisively in technological capacity. It seems natural to me to label (a) as “mistake theory” and (b) as “conflict theory”, and you have a philosophical disagreement here, but this seems like a quibble.
I agree that, as presented, adopting those object-level beliefs would seem to more naturally lend itself to a conflict theory view (vs. mistake theory), but my quibble is with the way you phrased your inference as to what view made most sense to adopt based on the presented facts as a claim that MIRI told you to adopt that view.
The account I give does support this. His assignment to me would not make sense as an argument of his proposition, that “the pieces to make AGI are already out there and someone just has to put them together”, unless he expected that in completing the assignment I would gain evidence for that proposition. For that to happen, it would have to be an AGI design that is workable (in the sense that with more compute and some fine-tuning, it would actually work); non-workable AGI designs are created all the time in the AI field, and provide no significant evidence about the proposition he was asserting.
Here is your summary: “I was given ridiculous statements and assignments including the claim that MIRI already knew about a working AGI design and that it would not be that hard for me to come up with a working AGI design on short notice just by thinking about it, without being given hints.”
Here is the anecdote: “I was told, by Nate Soares, that the pieces to make AGI are likely already out there and someone just has to put them together. He did not tell me anything about how to make such an AGI, on the basis that this would be dangerous. Instead, he encouraged me to figure it out for myself, saying it was within my abilities to do so. Now, I am not exactly bad at thinking about AGI; I had, before working at MIRI, gotten a Master’s degree at Stanford studying machine learning, and I had previously helped write a paper about combining probabilistic programming with machine learning. But figuring out how to create an AGI was and is so far beyond my abilities that this was a completely ridiculous expectation.”
Compare: “MIRI already knew about a working AGI design and that it would not be that hard for me to come up with a working AGI design on short notice just by thinking about it” with “the pieces to make AGI are likely already out there and someone just has to put them together… he encouraged me to figure it out for myself, saying it was within my abilities to do so”. The first is making several much stronger claims than the second (w.r.t. certainty & specific knowledge of a working design, and their belief in how difficult it would be for you to generate one yourself under what timeframe), in a way that makes MIRI/Nate seem much more unreasonable. If there are specific details omitted from the anecdote that support those stronger claims, I think it would make sense to include them; else we have something resembling a motte & bailey (specific, strong claim, supported by much weaker anecdote which in many possible worlds describes a totally reasonable interaction).
> Scott’s citation of research “showing greater mental modeling and verbal intelligence in relatives of schizophrenics” does not imply that Scott thinks it is a good idea to attempt to induce sub-clinical schizotypal states in people
Yes, I mentioned disagreement about tradeoffs later in the post.
If you agree that Scott & Michael have critical disagreements about the trade-offs in a way that’s relevant to the question at hand—more so than the surface-level agreement Scott’s writing demonstrates—why is this included at all? The implication that one might reasonably be expected to take from “His belief… is also shared by Scott Alexander and many academics.” is, in fact, more false than not.
I believe he was in the chat thread that made the decision, though I’m not sure.
Thanks for clarifying.
Later in the thread I give details about what claims I was actually making. Those claims are broadly consistent with materialist ontology.
Your original claim: “This increases the chance that someone like me could be psychiatrically incarcerated for talking about things that a substantial percentage of the general public (e.g. New Age people and Christians) talk about, and which could be explained in terms that don’t use magical concepts. This is inappropriately enforcing the norms of a minority ideological community as if they were widely accepted professional standards.”
I’m less concerned with the actual claims you were making at the time than with the fact that there is an extremely reasonable explanation for being concerned (in the general case, not in whatever your specific situation was) if a materialist starts taking seriously about auras and demons. That explanation is not that Scott was “enforcing the norms of a minority ideological community”.
It is massively surprising that the suicide was specifically narratized as being caused by inter-hemisphere conflict and precommitments/extortion. Precommitments/extortion are central LessWrong decision theory topics.
Why is this surprising? I agree that it would be surprising if it was narrativized that way by detached outside observers, given that they’d have had no particular reason to construct that sort of narrative, but given the intense focus those in question had on those subjects, as well as those close to them, I’m not sure why you’d be surprised that they ascribed the suicide to the novel self-experimentation they were doing immediately prior to the suicide, which led the person who committed suicide to publicly declare ahead of time that would be why they committed suicide, if they did. That seems like a totally reasonable narrative from their perspective. But, again, none of this information was widely known. I’ve read most of Ziz’s blog and even I didn’t know about the specific sequence of events you described. I’m still not surprised, though, because I place very little credence on the idea that Maia Pasek correctly reasoned her way into killing herself, starting from anything resembling a set of values I’d endorse.
I mentioned the specific example that I told a friend about it and then told them not to tell other people. I would guess that Ziz told others who also did this.
Ok, granted, you and (maybe) Ziz do seem to do the things you’re claiming the community at large does, as central rather than non-central behaviors. This does not justify the implication behind “which makes it strong evidence that people are indeed participating in this cover-up”.
Passive distributed conspiracy, directing attention away from critical details, is sufficient to hide important parts of the world from view.
I don’t even know what this is claiming. Who is directing attention away from what details? How deliberate is it? How do you distinguish this world from one where “people don’t talk about suicide because they didn’t know the person and generally don’t find much value in attempting to psychoanalyze people who committed suicide”?
Central people made efforts to convince Ziz that she was likely to be “net negative” due to her willingness to generally reveal information. Ziz’s own moral philosophy may imply that many “non-good” people are net negative. It is unsurprising if ideology influenced people towards suicide. The “being picked off” could come from mostly-unconscious motives.
I’m not following the chain of logic here. Ziz claims that Anna told her she thought Ziz was likely to be net negative (in the context of AI safety research), after Ziz directly asked her if she thought that. Are you claiming that Anna was sufficiently familiar with the details of Ziz’s ontology (which, afaik, she hadn’t even developed in any detail at that point?) to predict that it might tempt Ziz to commit suicide? Because I’m not seeing how else you get from “Anna answered a question that Ziz asked” to “people may be picked off one by one”.
The story of people being unwilling to confirm or deny whether they would investigate if a friend disappeared was very striking at the time, I was surprised how far people would go to avoid investigation.
I agree that the interaction, as described, sounds quite unusual, mostly due to the extended length where they refused to provide a concrete “yes or no” answer. I would be surprised if many of my friends would sincerely promise to investigate if I disappeared (though I can think of at least one who would), but I would be even more surprised if they refused to tell me whether or not they’d be willing to do so, in such a protracted manner.
No. But the magnitude of how bad they are depends on the specific details. I mentioned a blog post that says to imagine being dipped into lava. This was considered a relevant argument about the severity of s-risks.
I predict that if we anonymously polled MIRI researchers (or AI alignment researchers more broadly), very few of them would endorse “thinking about extreme AI torture scenarios [is] part of my job”, if it carries the implication that they also need to think about those scenarios in explicit detail rather than “many 0s, now put a minus sign in front of them”.
It wasn’t specifically suggested. It was implied by the general research landscape presented e.g. Eliezer’s Arbital article.
So it sounds like you agree that to the extent that it was part of your job “to imagine myself in the role of someone who is going to be creating the AI that could make everything literally the worst it could possibly be”, that was an inference you drew (maybe reasonably!) from the environment, rather than someone at MIRI telling you explicitly that it was part of your job (or strongly implying the same)? Again, to be clear, my issues are with the framing that makes it seem like these are things that other people did or said, rather than with whether or not these were locally valid inferences to be drawing.
They shouldn’t expect anyone to believe them if they don’t make persuasive arguments or do something similar. If I don’t agree with them that AI is likely to come soon, why would I do research based on that assumption? Presumably he would prefer me to, if in fact AGI is likely to come soon.
The original claim you were ridiculing was not that “AI is likely to come soon”.
He’s clearly giving “AI timelines are bad for one’s mental health” as a reason why his feelings about AI timelines may be unwise to voice.
Agreed, but what you said is this: “In a recent post, Eliezer Yudkowsky explicitly says that voicing “AGI timelines” is “not great for one’s mental health”, a new additional consideration for suppressing information about timelines.”
I even agree that you could reasonably interpret what he actually said (as opposed to what you said he said) as a reason to avoid discussing AI timelines, but, crucially, there is no reason to believe he intended that statement to be read that way, and it doesn’t support your claim that you were “constantly encouraged to think very carefully about the negative consequences of publishing anything about AI”. He is providing a reason why he himself does not make much of a habit of discussing AI timelines, and that reason is that he is worried about the mental health of others, not that he thinks discussing it pessimizes for timeline outcomes.
Part of our research was decision theory, so talking about decision theory would be one way to talk about the desirability of the security policies. Also, I mentioned Nate and Anna discouraging Michael Vassar from talking to researchers, which fits the pattern of preventing discussion of interactions with MIRI, as he was a notable critic of MIRI at that time.
This doesn’t seem responsive to my objection, which is the comparison to cult abuse tactics. Asking Michael Vassar to not talk to researchers (during work hours?) is very much not the same thing as asking researchers not to talk to each other. I agree that there are situations where asking Michael Vassar not to talk to researchers would have been inappropriate, but without details on the situation I can’t just nod my head and say, “yep, classic cult tactic”, and in any case this is not what you originally provided as evidence(?) of cult-like behavior.
I’ve corrected this due to jefftk making the same point.
Thanks.
I specifically said that other explanations offered were not convincing. Maybe they would convince you, but they did not convince me.
Ok, so this is just a highly specific claim about the internal motivations of other agents that doesn’t have any supporting evidence—not even what unconvincing arguments they offered.
I did not mean this as an argument against the claim “Michael Vassar seeks to induce psychotic states in people”. I meant his text to Zack as an argument against this claim. It is not perfect evidence, but Scott did not present significant positive evidence for the claim either.
Yes, I agree. My understanding of this post is that it’s substantially devoted to rebutting the argument that Michael Vassar was a meaningful contributor to your mental health problems. I think the fact that Michael Vassar directly interacted with you during the relevant timeframe in a way which you yourself think made things worse is notable, in the sense that for most plausible sets of priors, you should probably be updating upwards on the hypothesis that “spending time around Michael Vassar is more likely lead to psychosis than spending time around most other people”, irrespective of his state of knowledge & motivations at the time.
Not going to respond to all these, a lot seem like nitpicks.
Your original claim included the phrase “the only possible alternative hypothesis”, so this seems totally non-responsive to my problem with it.
My other point was that the problems were still experienced “at MIRI” even if they were caused by other things in the social environment.
That’s good to know, thanks. I think it would make your point here much stronger and more legible if those specific details were included in the original claim.
Edited.
Compare: “MIRI already knew about a working AGI design and that it would not be that hard for me to come up with a working AGI design on short notice just by thinking about it” with “the pieces to make AGI are likely already out there and someone just has to put them together… he encouraged me to figure it out for myself, saying it was within my abilities to do so”. The first is making several much stronger claims than the second (w.r.t. certainty & specific knowledge of a working design, and their belief in how difficult it would be for you to generate one yourself under what timeframe), in a way that makes MIRI/Nate seem much more unreasonable.
Nate implied he had already completed the assignment he was giving me.
The assignment wouldn’t provide evidence about whether the pieces to make AGI are already out there unless it was “workable” in the sense that iterative improvement with more compute and theory-light technique iteration would produce AGI.
If you agree that Scott & Michael have critical disagreements about the trade-offs in a way that’s relevant to the question at hand—more so than the surface-level agreement Scott’s writing demonstrates—why is this included at all?
Edited to make it clear that they disagree. The agreement is relevant to place a bound on the scope of what they actually disagree on.
I’m not following the chain of logic here. Ziz claims that Anna told her she thought Ziz was likely to be net negative (in the context of AI safety research), after Ziz directly asked her if she thought that. Are you claiming that Anna was sufficiently familiar with the details of Ziz’s ontology (which, afaik, she hadn’t even developed in any detail at that point?) to predict that it might tempt Ziz to commit suicide?
She might have guessed based on Ziz’s utilitarian futurism (this wouldn’t require knowing many specific details), or might not have been thinking about that consciously. It’s more likely she was trying to control Ziz (she has admitted to generally controlling people around CFAR by e.g. hoarding info). I think my general point is that people are trying to memetically compete with each other in ways that involve labeling others “net negative” in a way that people can very understandably internalize and which would lead to suicide. It’s more like a competition to drive each other insane than one to directly kill each other. A lot of competition (e.g. the kind that would be predicted by evolutionary theory) is subconscious and doesn’t indicate legal responsibility.
Anyway, I edited to make it clearer that many of the influences in question are subconscious and/or memetic.
I predict that if we anonymously polled MIRI researchers (or AI alignment researchers more broadly), very few of them would endorse “thinking about extreme AI torture scenarios [is] part of my job”, if it carries the implication that they also need to think about those scenarios in explicit detail rather than “many 0s, now put a minus sign in front of them”.
I predict that they would say that having some philosophical thoughts about negative utilitarianism and related considerations would be part of their job, and that AI torture scenarios are relevant to that, although perhaps not something they would specifically need to think about.
So it sounds like you agree that to the extent that it was part of your job “to imagine myself in the role of someone who is going to be creating the AI that could make everything literally the worst it could possibly be”, that was an inference you drew (maybe reasonably!) from the environment, rather than someone at MIRI telling you explicitly that it was part of your job (or strongly implying the same)?
Edited to make this clearer.
The original claim you were ridiculing was not that “AI is likely to come soon”.
They’re highly related, having a working AGI design is an argument for short timelines.
He is providing a reason why he himself does not make much of a habit of discussing AI timelines, and that reason is that he is worried about the mental health of others, not that he thinks discussing it pessimizes for timeline outcomes.
Sure, I mentioned it as a consideration other than the consideration I already mentioned about making AI come sooner.
I think the fact that Michael Vassar directly interacted with you during the relevant timeframe in a way which you yourself think made things worse is notable, in the sense that for most plausible sets of priors, you should probably be updating upwards on the hypothesis that “spending time around Michael Vassar is more likely lead to psychosis than spending time around most other people”, irrespective of his state of knowledge & motivations at the time.
I agree it’s weak evidence for that proposition. However the fact that he gave me useful philosophical advice is evidence against that proposition. In total the public info Scott and I have revealed provides very little directional evidence about this proposition.
I appreciate that you took the time to respond to my post in detail. I explained at the top why I had a difficult time engaging productively with your post (i.e. learning from it). I did learn some specific things, such as the claimed sequence of events prior to Maia’s suicide, and Nate’s recent retraction of his earlier public statement on OpenAI. Those are things which are either unambiguously claims about reality, or have legible evidence supporting them.
None of these carry the same implication that the community, centrally, was engaging in the claimed concealment. This phrasing deflects agency away from the person performing the action, and on to community leaders: “Ziz labeling it as an infohazard is in compliance with feedback Ziz has received from community leaders.” “Ziz labeled something that might have contributed to Maia’s suicide an infohazard, possibly as a result of feedback she got from someone else in the community well before she shared that information with Maia” implies something very different from “Many others have worked to conceal the circumstances of their deaths”, which in context makes it sound like an active conspiracy engaged in by central figures in the community. People not drawing attention to Fluttershy’s post, even after you posted about it, is not active concealment.
Your original claim included the phrase “the only possible alternative hypothesis”, so this seems totally non-responsive to my problem with it.
Again, this seems non-responsive to what I’m saying is the issue, which is that the “general view” is more or less useless for evaluating how much in “agreement” they really are, as opposed to the specific details.
That’s good to know, thanks. I think it would make your point here much stronger and more legible if those specific details were included in the original claim.
I agree that, as presented, adopting those object-level beliefs would seem to more naturally lend itself to a conflict theory view (vs. mistake theory), but my quibble is with the way you phrased your inference as to what view made most sense to adopt based on the presented facts as a claim that MIRI told you to adopt that view.
Here is your summary: “I was given ridiculous statements and assignments including the claim that MIRI already knew about a working AGI design and that it would not be that hard for me to come up with a working AGI design on short notice just by thinking about it, without being given hints.”
Here is the anecdote: “I was told, by Nate Soares, that the pieces to make AGI are likely already out there and someone just has to put them together. He did not tell me anything about how to make such an AGI, on the basis that this would be dangerous. Instead, he encouraged me to figure it out for myself, saying it was within my abilities to do so. Now, I am not exactly bad at thinking about AGI; I had, before working at MIRI, gotten a Master’s degree at Stanford studying machine learning, and I had previously helped write a paper about combining probabilistic programming with machine learning. But figuring out how to create an AGI was and is so far beyond my abilities that this was a completely ridiculous expectation.”
Compare: “MIRI already knew about a working AGI design and that it would not be that hard for me to come up with a working AGI design on short notice just by thinking about it” with “the pieces to make AGI are likely already out there and someone just has to put them together… he encouraged me to figure it out for myself, saying it was within my abilities to do so”. The first is making several much stronger claims than the second (w.r.t. certainty & specific knowledge of a working design, and their belief in how difficult it would be for you to generate one yourself under what timeframe), in a way that makes MIRI/Nate seem much more unreasonable. If there are specific details omitted from the anecdote that support those stronger claims, I think it would make sense to include them; else we have something resembling a motte & bailey (specific, strong claim, supported by much weaker anecdote which in many possible worlds describes a totally reasonable interaction).
If you agree that Scott & Michael have critical disagreements about the trade-offs in a way that’s relevant to the question at hand—more so than the surface-level agreement Scott’s writing demonstrates—why is this included at all? The implication that one might reasonably be expected to take from “His belief… is also shared by Scott Alexander and many academics.” is, in fact, more false than not.
Thanks for clarifying.
Your original claim: “This increases the chance that someone like me could be psychiatrically incarcerated for talking about things that a substantial percentage of the general public (e.g. New Age people and Christians) talk about, and which could be explained in terms that don’t use magical concepts. This is inappropriately enforcing the norms of a minority ideological community as if they were widely accepted professional standards.”
I’m less concerned with the actual claims you were making at the time than with the fact that there is an extremely reasonable explanation for being concerned (in the general case, not in whatever your specific situation was) if a materialist starts taking seriously about auras and demons. That explanation is not that Scott was “enforcing the norms of a minority ideological community”.
Why is this surprising? I agree that it would be surprising if it was narrativized that way by detached outside observers, given that they’d have had no particular reason to construct that sort of narrative, but given the intense focus those in question had on those subjects, as well as those close to them, I’m not sure why you’d be surprised that they ascribed the suicide to the novel self-experimentation they were doing immediately prior to the suicide, which led the person who committed suicide to publicly declare ahead of time that would be why they committed suicide, if they did. That seems like a totally reasonable narrative from their perspective. But, again, none of this information was widely known. I’ve read most of Ziz’s blog and even I didn’t know about the specific sequence of events you described. I’m still not surprised, though, because I place very little credence on the idea that Maia Pasek correctly reasoned her way into killing herself, starting from anything resembling a set of values I’d endorse.
Ok, granted, you and (maybe) Ziz do seem to do the things you’re claiming the community at large does, as central rather than non-central behaviors. This does not justify the implication behind “which makes it strong evidence that people are indeed participating in this cover-up”.
I don’t even know what this is claiming. Who is directing attention away from what details? How deliberate is it? How do you distinguish this world from one where “people don’t talk about suicide because they didn’t know the person and generally don’t find much value in attempting to psychoanalyze people who committed suicide”?
I’m not following the chain of logic here. Ziz claims that Anna told her she thought Ziz was likely to be net negative (in the context of AI safety research), after Ziz directly asked her if she thought that. Are you claiming that Anna was sufficiently familiar with the details of Ziz’s ontology (which, afaik, she hadn’t even developed in any detail at that point?) to predict that it might tempt Ziz to commit suicide? Because I’m not seeing how else you get from “Anna answered a question that Ziz asked” to “people may be picked off one by one”.
I agree that the interaction, as described, sounds quite unusual, mostly due to the extended length where they refused to provide a concrete “yes or no” answer. I would be surprised if many of my friends would sincerely promise to investigate if I disappeared (though I can think of at least one who would), but I would be even more surprised if they refused to tell me whether or not they’d be willing to do so, in such a protracted manner.
I predict that if we anonymously polled MIRI researchers (or AI alignment researchers more broadly), very few of them would endorse “thinking about extreme AI torture scenarios [is] part of my job”, if it carries the implication that they also need to think about those scenarios in explicit detail rather than “many 0s, now put a minus sign in front of them”.
So it sounds like you agree that to the extent that it was part of your job “to imagine myself in the role of someone who is going to be creating the AI that could make everything literally the worst it could possibly be”, that was an inference you drew (maybe reasonably!) from the environment, rather than someone at MIRI telling you explicitly that it was part of your job (or strongly implying the same)? Again, to be clear, my issues are with the framing that makes it seem like these are things that other people did or said, rather than with whether or not these were locally valid inferences to be drawing.
The original claim you were ridiculing was not that “AI is likely to come soon”.
Agreed, but what you said is this: “In a recent post, Eliezer Yudkowsky explicitly says that voicing “AGI timelines” is “not great for one’s mental health”, a new additional consideration for suppressing information about timelines.”
I even agree that you could reasonably interpret what he actually said (as opposed to what you said he said) as a reason to avoid discussing AI timelines, but, crucially, there is no reason to believe he intended that statement to be read that way, and it doesn’t support your claim that you were “constantly encouraged to think very carefully about the negative consequences of publishing anything about AI”. He is providing a reason why he himself does not make much of a habit of discussing AI timelines, and that reason is that he is worried about the mental health of others, not that he thinks discussing it pessimizes for timeline outcomes.
This doesn’t seem responsive to my objection, which is the comparison to cult abuse tactics. Asking Michael Vassar to not talk to researchers (during work hours?) is very much not the same thing as asking researchers not to talk to each other. I agree that there are situations where asking Michael Vassar not to talk to researchers would have been inappropriate, but without details on the situation I can’t just nod my head and say, “yep, classic cult tactic”, and in any case this is not what you originally provided as evidence(?) of cult-like behavior.
Thanks.
Ok, so this is just a highly specific claim about the internal motivations of other agents that doesn’t have any supporting evidence—not even what unconvincing arguments they offered.
Yes, I agree. My understanding of this post is that it’s substantially devoted to rebutting the argument that Michael Vassar was a meaningful contributor to your mental health problems. I think the fact that Michael Vassar directly interacted with you during the relevant timeframe in a way which you yourself think made things worse is notable, in the sense that for most plausible sets of priors, you should probably be updating upwards on the hypothesis that “spending time around Michael Vassar is more likely lead to psychosis than spending time around most other people”, irrespective of his state of knowledge & motivations at the time.
Not going to respond to all these, a lot seem like nitpicks.
My other point was that the problems were still experienced “at MIRI” even if they were caused by other things in the social environment.
Edited.
Nate implied he had already completed the assignment he was giving me.
The assignment wouldn’t provide evidence about whether the pieces to make AGI are already out there unless it was “workable” in the sense that iterative improvement with more compute and theory-light technique iteration would produce AGI.
Edited to make it clear that they disagree. The agreement is relevant to place a bound on the scope of what they actually disagree on.
She might have guessed based on Ziz’s utilitarian futurism (this wouldn’t require knowing many specific details), or might not have been thinking about that consciously. It’s more likely she was trying to control Ziz (she has admitted to generally controlling people around CFAR by e.g. hoarding info). I think my general point is that people are trying to memetically compete with each other in ways that involve labeling others “net negative” in a way that people can very understandably internalize and which would lead to suicide. It’s more like a competition to drive each other insane than one to directly kill each other. A lot of competition (e.g. the kind that would be predicted by evolutionary theory) is subconscious and doesn’t indicate legal responsibility.
Anyway, I edited to make it clearer that many of the influences in question are subconscious and/or memetic.
I predict that they would say that having some philosophical thoughts about negative utilitarianism and related considerations would be part of their job, and that AI torture scenarios are relevant to that, although perhaps not something they would specifically need to think about.
Edited to make this clearer.
They’re highly related, having a working AGI design is an argument for short timelines.
Sure, I mentioned it as a consideration other than the consideration I already mentioned about making AI come sooner.
I agree it’s weak evidence for that proposition. However the fact that he gave me useful philosophical advice is evidence against that proposition. In total the public info Scott and I have revealed provides very little directional evidence about this proposition.