(Please don’t upvote this comment till you’ve read it fully; I’m interpreting upvotes in a specific way.) Question for anyone on LW: If I had a viable preliminary Friendly AI research program, aimed largely at doing the technical analysis necessary to determine as well as possible the feasibility and difficulty of Friendly AI for various values of “Friendly”, and wrote clearly and concretely about the necessary steps in pursuing this analysis, and listed and described a small number of people (less than 5, but how many could actually be convinced to focus on doing the analysis would depend on funds) who I know of who could usefully work on such an analysis, and committed to have certain summaries published online at various points (after actually considering concrete possibilities for failure, planning fallacy, etc., like real rationalists should), and associated with a few (roughly 5) high status people (people like Anders Sandberg or Max Tegmark, e.g. by convincing them to be on an advisory board), would this have a decent chance of causing you or someone you know to donate $100 or more to support this research program? (I have a weird rather mixed reputation among the greater LW community, so if that affects you negatively please pretend that someone with a more solid reputation but without super high karma is asking this question, like Steven Kaas.) You can upvote for “yes” and comment about any details, e.g. if you know someone who would possibly donate significantly more than $100. (Please don’t downvote for “no”, ’cuz that’s the default answer and will drown out any “yes” ones.)
I have a weird rather mixed reputation among the greater LW community, so if that affects you negatively please pretend that someone with a more solid reputation but without super high karma is asking this question, like Steven Kaas.
Unless you would be much less involved in this potential program than the comment indicates, this seems like an inappropriate request. If people view you negatively due to your posting history, they should absolutely take that information into account in assessing how likely they would be to provide financial support to such a program (assuming that the negative view is based on relevant considerations such as your apparent communication or reasoning skills as demonstrated in your comments).
Fair enough, but in light of your phrasing in both the original comment (“If I [did the following things]”) and your comment immediately following it (quoted below; emphasis added), it certainly appeared to me that you seemed to be describing a significant role for yourself, even though your proposal was general overall.
(Some people, including me, would really like it if a competent and FAI-focused uber-rationalist non-profit existed. I know people who will soon have enough momentum to make this happen. I am significantly more familiar with the specifics of FAI (and of hardcore SingInst-style rationality) than many of those people and almost anyone else in the world, so it’d be necessary that I put a lot of hours into working with those who are higher status than me and better at getting things done but less familiar with technical Friendliness. But I have many other things I could be doing. Hence the question.)
Sorry, could you say again what exactly you want to do? I mean, what’s the output here that the money is paying for; a Friendly AI, a theory that can be used to construct a Friendly AI, or an analysis that purports to say whether or not Friendly AI is “feasible”, or what?
Money would pay for marginal output, e.g. in the form of increased collaboration, I think, since the best Friendliness-cognizant x-rationalists would likely already be working on similar things.
I was trying to quickly gauge vague interest in a vague notion. I think that my original comment was at roughly the most accurate and honest level of vagueness (i.e. “aimed largely [i.e. primarily] at doing the technical analysis necessary to determine as well as possible the feasibility and difficulty [e.g. how many Von Neumanns, Turings, and/or Aristotles would it take?] of Friendly AI for various (logical) probabilities of Friendliness [e.g. is the algorithm meta-reflective enough to fall into (one of) some imagined Friendliness attractor basin(s)?]”). Value of information regarding difficulty of Friendly-ish AI is high, but research into that question is naturally tied to Friendly AI theory itself. I’m thinking… Goedel machine stability more than ambient decision theory, history of computation more than any kind of validity semantics. To some extent it depends on who plans to actually work on what stuff from the open problems lists. There are many interesting technical threads that people might start pulling on soon, and it’s unclear to me to what extent they actually will pull on them or to what extent pulling on them will give us a better sense of the problem.
[Stuff it would take too many paragraphs to explain why it’s worth pointing out specifically:] Theory of justification seems to be roughly as developed as theory of computation was before the advent of Leibniz; Leibniz saw a plethora of connections between philosophy, symbolic logic, and engineering and thus developed some correctly thematically centered proto-theory. I’m trying to make a Leibniz, and hopefully SingInst can make a Turing. (Two other roughly analogous historical conceptual advances are natural selection and temperature.)
Well, my probability that you could or would do anything useful, given money, just dropped straight off a cliff. But perhaps you’re just having trouble communicating. That is to say: What the hell are you talking about.
If you’re going to ask for money on LW, plain English response, please: What’s the output here that the money is paying for; (1) a Friendly AI, (2) a theory that can be used to construct a Friendly AI, or (3) an analysis that purports to say whether or not Friendly AI is “feasible”? Please pick one of the pre-written options; I now doubt your ability to write your response ab initio.
Dude, it’s right there: “feasibility and difficulty”, in this sentence which I am now repeating for the second time:
aimed largely [i.e. primarily] at doing the technical analysis necessary to determine as well as possible the feasibility and difficulty [e.g. how many Von Neumanns, Turings, and/or Aristotles would it take?] of Friendly AI for various (logical) probabilities of Friendliness [e.g. is the algorithm meta-reflective enough to fall into (one of) some imagined Friendliness attractor basin(s)?]”).
(Bold added for emphasis, annotations in [brackets] were in the original.)
The next sentence:
Value of information regarding difficulty of Friendly-ish AI is high, but research into that question is naturally tied to Friendly AI theory itself.
Or if you really need it spelled out for you again and again, the output would primarily be (3) but secondarily (2) as you need some of (2) to do (3).
Because you clearly need things pointed out multiple times, I’ll remind you that I put my response in the original comment that you originally responded to, without the later clarifications that I’d put in for apparently no one’s benefit:
If I had a viable preliminary Friendly AI research program, aimed largely at doing the technical analysis necessary to determine as well as possible the feasibility and difficulty of Friendly AI for various values of “Friendly” [...]
(Those italics were in the original comment!)
If you’re going to ask for money on LW
I wasn’t asking for money on Less Wrong! As I said, “I was trying to quickly gauge vague interest in a vague notion.” What the hell are you talking about.
I now doubt your ability to write your response ab initio.
I’ve doubted your ability to read for a long time, but this is pretty bad. The sad thing is you’re probably not doing this intentionally.
I think the problem here is that your posting style, to be frank, often obscures your point.
In most cases, posts that consist of a to-the-point answer followed by longer explanations use the initial statement to make a concise case. For instance, in this post, my first sentence sums up what I think about the situation and the rest explains that thought in more detail so as to convey a more nuanced impression.
By contrast, when Eliezer asked “What’s the output here that the money is paying for,” your first sentence was “Money would pay for marginal output, e.g. in the form of increased collaboration, I think, since the best Friendliness-cognizant x-rationalists would likely already be working on similar things.” This does not really answer his question, and while you clarify this with your later points, the overall message is garbled.
The fact that your true answer is buried in the middle of a paragraph does not really help things much. Though I can see what you are trying to say, I can’t in good and honest conscience describe it as clear. Had you answered, on the other hand, “Money would pay for the technical analysis necessary to determine as well as possible the feasibility and difficulty of FAI...” as your first sentence, I think your post would have been more clear and more likely to be understood.
The sentences I put before the direct answer to Eliezer’s question were meant to correct some of Eliezer’s misapprehensions that were more fundamental than the object of his question. Eliezer’s infamous for uncharitably misinterpreting people and it was clear he’d misinterpreted some key aspects of my original comment, e.g. my purpose in writing it. If I’d immediately directly answered his question that would have been dishonest; it would have contributed further to his having a false view of what I was actually talking about. Less altruistically it would be like I was admitting to his future selves or to external observers that I agreed that his model of my purposes was accurate and that this model could legitimately be used to assert that I was unjustified in any of many possible ways. Thus I briefly (a mere two sentences) attempted to address what seemed likely to be Eliezer’s underlying confusions before addressing his object level question. (Interestingly Eliezer does this quite often, but unfortunately he often assumes people are confused in ways that they are not.)
Given these constraints, what should I have done? In retrospect I should have gone meta, of course, like always. What else?
Given those constraints, I would probably write something like “Money would pay for marginal output in the form of increased collaboration on the technical analysis necessary to determine as well as possible the feasibility and difficulty of FAI” for my first sentence and elaborate as strictly necessary. That seems rather more cumbersome than I’d like, but it’s also a lot of information to try and convey in one sentence!
Alternatively, I would consider something along the lines of “Money would pay for the technical analysis necessary to determine as well as possible the feasibility and difficulty of FAI, but not directly—since the best Friendliness-cognizant x-rationalists would likely already be working on similar things, the money would go towards setting up better communication, coordination, and collaboration for that group.”
That said, I am unaware of any reputation Eliezer has in the field of interpreting people, and personally haven’t received the impression that he’s consistently unusually bad or uncharitable at it. Then again, I have something of a reputation—at least in person—for being too charitable, so perhaps I’m being too light on Eliezer (or you?) here.
I think the problem here is that your posting style, to be frank, often obscures your point.
I acknowledge this. But it seems to me that the larger problem is that Eliezer simply doesn’t know how to read what people actually say. Less Wrong mostly doesn’t either, and humans in general certainly don’t. This is a very serious problem with LW-style rationality (and with humanity). There are extremely talented rationalists who do not have this problem; it is an artifact of Eliezer’s psychology and not of the art of rationality.
It’s hardly fair to blame the reader when you’ve got “sentences” like this:
I think that my original comment was at roughly the most accurate and honest level of vagueness (i.e. “aimed largely [i.e. primarily] at doing the technical analysis necessary to determine as well as possible the feasibility and difficulty [e.g. how many Von Neumanns, Turings, and/or Aristotles would it take?] of Friendly AI for various (logical) probabilities of Friendliness [e.g. is the algorithm meta-reflective enough to fall into (one of) some imagined Friendliness attractor basin(s)?]”).
That was the second version of the sentence, the first one had much clearer syntax and even italicized the answer to Eliezer’s subsequent question. It looks the way it does because Eliezer apparently couldn’t extract meaning out of my original sentence despite it clearly answering his question, so I tried to expand on the relevant points with bracketed concrete examples. Here’s the original:
If I had a viable preliminary Friendly AI research program, aimed largely at doing the technical analysis necessary to determine as well as possible the feasibility and difficulty of Friendly AI for various values of “Friendly” [...]
What you say might be true, but this one example is negligible compared to the mountain of other evidence concerning inability to read much more important things (which are unrelated to me). I won’t give that evidence here.
If that’s indeed the case (I haven’t noticed this flaw myself), I suggest that you write articles (or perhaps commission/petition others to have them written) describing this flaw and how to correct it. Eliminating such a flaw or providing means of averting it would greatly aid LW and the community in general.
Unfortunately that is not currently possible for many reasons, including some large ones I can’t talk about and that I can’t talk about why I can’t talk about. I can’t see any way that it would become possible in the next few years either. I find this stressful; it’s why I make token attempts to communicate in extremely abstract or indirect ways with Less Wrong, despite the apparent fruitlessness. But there’s really nothing for it.
Unrelated public announcement: People who go back and downvote every comment someone’s made, please, stop doing that. It’s a clever way to pull information cascades in your direction but it is clearly an abuse of the content filtering system and highly dishonorable. If you truly must use such tactics, downvoting a few of your enemy’s top level posts is much less evil; your enemy loses the karma and takes the hint without your severely biasing the public perception of your enemy’s standard discourse. Please.
(I just lost 150 karma points in a few minutes and that’ll probably continue for awhile. This happens a lot.)
Unfortunately that is not currently possible for many reasons, including some large ones I can’t talk about and that I can’t talk about why I can’t talk about.
Why can’t you talk about why you can’t talk about them?
I’m not a big fan of the appeal to secret reasons, so I think I’m going to have pull out of this discussion. I will note, however, that you personally seem to be involved in more misunderstandings than the average LW poster, so while it’s certainly possible that your secret reasons are true and valid and Eliezer just sucks at reading or whatever, you may want to clarify certain elements of your own communication as well.
I unfortunately predict that “going more meta” will not be strongly received here.
Unfortunately that is not currently possible for many reasons, including some large ones I can’t talk about and that I can’t talk about why I can’t talk about.
Are we still talking about improving general reading comprehension? What could possibly be dangerous about that?
(Some people, including me, would really like it if a competent and FAI-focused uber-rationalist non-profit existed. I know people who will soon have enough momentum to make this happen. I am significantly more familiar with the specifics of FAI (and of hardcore SingInst-style rationality) than many of those people and almost anyone else in the world, so it’d be necessary that I put a lot of hours into working with those who are higher status than me and better at getting things done but less familiar with technical Friendliness. But I have many other things I could be doing. Hence the question.)
Does “FAI-focused” mean what I called code first? What are your thoughts on that post and its followup? What is this new non-profit planning to do differently from SIAI and why? What are the other things that you could be doing?
Jah. Well, at least determining whether or not “code first” is even reasonable, yeah, which is a difficult question in itself and only partially tied in with making direct progress on FAI.
What are your thoughts on that post and its followup?
You seem to have missed Oracle AI? (Eliezer’s dismissal of it isn’t particularly meaningful.) I agree with your concerns. This is why the main focus would at least initially be determining whether or not “code first” is a plausible approach (difficulty-wise and safety-wise). The value of information on that question is incredibly high and as you’ve pointed out it has not been sufficiently researched.
What is this new non-profit planning to do differently from SIAI and why?
Basically everything. SingInst is focused on funding a large research program and gaining the prestige necessary to influence (academic) culture and academic and political policy. They’re not currently doing any research on Friendly AI, and their political situation is such that I don’t expect them to be able to do so effectively for a while, if ever. I will not clarify this. (Actually their research associates are working on FAI-related things, but SingInst doesn’t pay them to do that.)
What are the other things that you could be doing?
Learning, mostly. Working with an unnamed group of x-risk-cognizant people that LW hasn’t heard of, in a way unrelated to their setting up a non-profit.
They’re not currently doing any research on Friendly AI, and their political situation is such that I don’t expect them to be able to do so effectively for a while, if ever.
My understanding is that SIAI recently tried to set up a new in-house research team to do preliminary research into FAI (i.e., not try to build an FAI yet, but just do whatever research that might be eventually helpful to that project). This effort didn’t get off the ground, but my understanding again is that it was because the researchers they tried to recruit had various reasons for not joining SIAI at this time. I was one of those they tried to recruit, and while I don’t know what the others’ reasons were, mine were mostly personal and not related to politics.
You must also know all this, since you were involved in this effort. So I’m confused why you say SIAI won’t be doing effective research on FAI due to its “political situation”. Did the others not join SIAI because they thought SIAI was in a bad political situation? (This seems unlikely to me.) Or are you referring to the overall lack of qualified, recruitable researchers as a “political situation”? If you are, why do you think this new organization would be able to do better?
(Or did you perhaps not learn the full story, and thought SIAI stopped this effort for political reasons?)
The answer to your question isn’t among your list of possible answers. The recent effort to start an in-house research team was a good attempt and didn’t fail for political reasons. I am speaking of other things. However I want to take a few weeks off from discussion of such topics; I seem to have given off the entirely wrong impression and would prefer to start such discussion anew in a better context, e.g. one that better emphasizes cooperation and tentativeness rather than reactionary competition. My apologies.
I was trying to quickly gauge vague interest in a vague notion.
I won’t give that evidence here.
(I won’t substantiate that claim here.)
I will not clarify this.
The answer to your question isn’t among your list of possible answers.
I find this stressful; it’s why I make token attempts to communicate in extremely abstract or indirect ways with Less Wrong, despite the apparent fruitlessness. But there’s really nothing for it.
It’s a good heuristic, but can be very psychologically difficult. E.g. if you think that not even trying to communicate will be seen as unjustified in retrospect even if people should know that there was no obvious way for you to communicate. This has happened enough to me that the thought of just giving up on communication is highly aversive; my fear of being blamed for not preventing others to take unjustified actions (that will cause me, them, and the universe counterfactually-needless pain) is too great. But whatever, I’m starting to get over it.
Like, I remember a pigeon dying… people dying… a girl who starved herself… a girl who cut herself… a girl who wanted to commit suicide… just, trust me, there are reasons that I’m afraid. I could talk about those reasons but I’d rather not. It’s just, if you don’t even make a token gesture it’s like you don’t even care at all, and it’s easier to be unjustified in a way that can be made to look sorta like caring than in a way that looks like thoughtlessness or carelessness.
(ETA: A lot of the time when people give me or others advice I mentally translate it to “The solution is simple, just shut up and be evil.”.)
I have no particular attachments to SIAI and would love to see a more effective Singularitarian organization formed if that were possible. I’m just having genuine trouble understanding why you think this new proposed organization will be able to do more effective FAI research. Perhaps you could use these few weeks off to ask some trusted advisors how to better communicate this point. (I understand you have sensitive information that you can’t reveal, but I’m guessing that you can do better even within that constraint.)
Perhaps you could use these few weeks off to ask some trusted advisors how to better communicate this point.
This is exactly the strategy I’ve decided to enact, e.g. talking to Anna. Thanks for being… gentle, I suppose? I’ve been getting a lot of flak lately, it’s nice to get some non-insulting advice sometimes. :)
(Somehow I completely failed to communicate the tentativeness of the ideas I was throwing around; in my head I was giving it about a 3% chance that I’d actually work on helping build an organization but I seem to have given off an impression of about 30%. I think this caused everyone’s brains to enter politics mode, which is not a good mode for brains to be in.)
I have no particular attachments to SIAI and would love to see a more effective Singularitarian organization formed if that were possible.
It’s rather strange how the SIAI is secretive. The military projects are secretive, the commercial projects are secretive alas: so few value transparency. An open project would surely do better, through being more obviously trustworthy and accountable, being better able to use talent across the internet, etc. I figure if the SIAI persists in not getting to grips with this issue, some other organisation will.
(Please don’t upvote this comment till you’ve read it fully; I’m interpreting upvotes in a specific way.) Question for anyone on LW: If I had a viable preliminary Friendly AI research program, aimed largely at doing the technical analysis necessary to determine as well as possible the feasibility and difficulty of Friendly AI for various values of “Friendly”, and wrote clearly and concretely about the necessary steps in pursuing this analysis, and listed and described a small number of people (less than 5, but how many could actually be convinced to focus on doing the analysis would depend on funds) who I know of who could usefully work on such an analysis, and committed to have certain summaries published online at various points (after actually considering concrete possibilities for failure, planning fallacy, etc., like real rationalists should), and associated with a few (roughly 5) high status people (people like Anders Sandberg or Max Tegmark, e.g. by convincing them to be on an advisory board), would this have a decent chance of causing you or someone you know to donate $100 or more to support this research program? (I have a weird rather mixed reputation among the greater LW community, so if that affects you negatively please pretend that someone with a more solid reputation but without super high karma is asking this question, like Steven Kaas.) You can upvote for “yes” and comment about any details, e.g. if you know someone who would possibly donate significantly more than $100. (Please don’t downvote for “no”, ’cuz that’s the default answer and will drown out any “yes” ones.)
Unless you would be much less involved in this potential program than the comment indicates, this seems like an inappropriate request. If people view you negatively due to your posting history, they should absolutely take that information into account in assessing how likely they would be to provide financial support to such a program (assuming that the negative view is based on relevant considerations such as your apparent communication or reasoning skills as demonstrated in your comments).
I was more interested in Less Wrong’s interest in new FAI-focused organizations generally than in anything particularly tied to me.
Fair enough, but in light of your phrasing in both the original comment (“If I [did the following things]”) and your comment immediately following it (quoted below; emphasis added), it certainly appeared to me that you seemed to be describing a significant role for yourself, even though your proposal was general overall.
Sorry, could you say again what exactly you want to do? I mean, what’s the output here that the money is paying for; a Friendly AI, a theory that can be used to construct a Friendly AI, or an analysis that purports to say whether or not Friendly AI is “feasible”, or what?
Money would pay for marginal output, e.g. in the form of increased collaboration, I think, since the best Friendliness-cognizant x-rationalists would likely already be working on similar things.
I was trying to quickly gauge vague interest in a vague notion. I think that my original comment was at roughly the most accurate and honest level of vagueness (i.e. “aimed largely [i.e. primarily] at doing the technical analysis necessary to determine as well as possible the feasibility and difficulty [e.g. how many Von Neumanns, Turings, and/or Aristotles would it take?] of Friendly AI for various (logical) probabilities of Friendliness [e.g. is the algorithm meta-reflective enough to fall into (one of) some imagined Friendliness attractor basin(s)?]”). Value of information regarding difficulty of Friendly-ish AI is high, but research into that question is naturally tied to Friendly AI theory itself. I’m thinking… Goedel machine stability more than ambient decision theory, history of computation more than any kind of validity semantics. To some extent it depends on who plans to actually work on what stuff from the open problems lists. There are many interesting technical threads that people might start pulling on soon, and it’s unclear to me to what extent they actually will pull on them or to what extent pulling on them will give us a better sense of the problem.
[Stuff it would take too many paragraphs to explain why it’s worth pointing out specifically:] Theory of justification seems to be roughly as developed as theory of computation was before the advent of Leibniz; Leibniz saw a plethora of connections between philosophy, symbolic logic, and engineering and thus developed some correctly thematically centered proto-theory. I’m trying to make a Leibniz, and hopefully SingInst can make a Turing. (Two other roughly analogous historical conceptual advances are natural selection and temperature.)
Well, my probability that you could or would do anything useful, given money, just dropped straight off a cliff. But perhaps you’re just having trouble communicating. That is to say: What the hell are you talking about.
If you’re going to ask for money on LW, plain English response, please: What’s the output here that the money is paying for; (1) a Friendly AI, (2) a theory that can be used to construct a Friendly AI, or (3) an analysis that purports to say whether or not Friendly AI is “feasible”? Please pick one of the pre-written options; I now doubt your ability to write your response ab initio.
That was amusingly written, but probably too harsh. You want people to like you, even if it’s only so they say nice things about you.
Dude, it’s right there: “feasibility and difficulty”, in this sentence which I am now repeating for the second time:
(Bold added for emphasis, annotations in [brackets] were in the original.)
The next sentence:
Or if you really need it spelled out for you again and again, the output would primarily be (3) but secondarily (2) as you need some of (2) to do (3).
Because you clearly need things pointed out multiple times, I’ll remind you that I put my response in the original comment that you originally responded to, without the later clarifications that I’d put in for apparently no one’s benefit:
(Those italics were in the original comment!)
I wasn’t asking for money on Less Wrong! As I said, “I was trying to quickly gauge vague interest in a vague notion.” What the hell are you talking about.
I’ve doubted your ability to read for a long time, but this is pretty bad. The sad thing is you’re probably not doing this intentionally.
I think the problem here is that your posting style, to be frank, often obscures your point.
In most cases, posts that consist of a to-the-point answer followed by longer explanations use the initial statement to make a concise case. For instance, in this post, my first sentence sums up what I think about the situation and the rest explains that thought in more detail so as to convey a more nuanced impression.
By contrast, when Eliezer asked “What’s the output here that the money is paying for,” your first sentence was “Money would pay for marginal output, e.g. in the form of increased collaboration, I think, since the best Friendliness-cognizant x-rationalists would likely already be working on similar things.” This does not really answer his question, and while you clarify this with your later points, the overall message is garbled.
The fact that your true answer is buried in the middle of a paragraph does not really help things much. Though I can see what you are trying to say, I can’t in good and honest conscience describe it as clear. Had you answered, on the other hand, “Money would pay for the technical analysis necessary to determine as well as possible the feasibility and difficulty of FAI...” as your first sentence, I think your post would have been more clear and more likely to be understood.
The sentences I put before the direct answer to Eliezer’s question were meant to correct some of Eliezer’s misapprehensions that were more fundamental than the object of his question. Eliezer’s infamous for uncharitably misinterpreting people and it was clear he’d misinterpreted some key aspects of my original comment, e.g. my purpose in writing it. If I’d immediately directly answered his question that would have been dishonest; it would have contributed further to his having a false view of what I was actually talking about. Less altruistically it would be like I was admitting to his future selves or to external observers that I agreed that his model of my purposes was accurate and that this model could legitimately be used to assert that I was unjustified in any of many possible ways. Thus I briefly (a mere two sentences) attempted to address what seemed likely to be Eliezer’s underlying confusions before addressing his object level question. (Interestingly Eliezer does this quite often, but unfortunately he often assumes people are confused in ways that they are not.)
Given these constraints, what should I have done? In retrospect I should have gone meta, of course, like always. What else?
Thanks much for the critique.
Given those constraints, I would probably write something like “Money would pay for marginal output in the form of increased collaboration on the technical analysis necessary to determine as well as possible the feasibility and difficulty of FAI” for my first sentence and elaborate as strictly necessary. That seems rather more cumbersome than I’d like, but it’s also a lot of information to try and convey in one sentence!
Alternatively, I would consider something along the lines of “Money would pay for the technical analysis necessary to determine as well as possible the feasibility and difficulty of FAI, but not directly—since the best Friendliness-cognizant x-rationalists would likely already be working on similar things, the money would go towards setting up better communication, coordination, and collaboration for that group.”
That said, I am unaware of any reputation Eliezer has in the field of interpreting people, and personally haven’t received the impression that he’s consistently unusually bad or uncharitable at it. Then again, I have something of a reputation—at least in person—for being too charitable, so perhaps I’m being too light on Eliezer (or you?) here.
I acknowledge this. But it seems to me that the larger problem is that Eliezer simply doesn’t know how to read what people actually say. Less Wrong mostly doesn’t either, and humans in general certainly don’t. This is a very serious problem with LW-style rationality (and with humanity). There are extremely talented rationalists who do not have this problem; it is an artifact of Eliezer’s psychology and not of the art of rationality.
It’s hardly fair to blame the reader when you’ve got “sentences” like this:
That was the second version of the sentence, the first one had much clearer syntax and even italicized the answer to Eliezer’s subsequent question. It looks the way it does because Eliezer apparently couldn’t extract meaning out of my original sentence despite it clearly answering his question, so I tried to expand on the relevant points with bracketed concrete examples. Here’s the original:
(emphasis in original)
Which starts with the word ‘if’ and fails to have a ‘then’.
If you took out ‘If I had’ and replaced it with ‘I would create’, then maybe it would be more in line with what you’re trying to say?
What you say might be true, but this one example is negligible compared to the mountain of other evidence concerning inability to read much more important things (which are unrelated to me). I won’t give that evidence here.
Certainly true, but that only means that we need to spend more effort on being as clear as possible.
If that’s indeed the case (I haven’t noticed this flaw myself), I suggest that you write articles (or perhaps commission/petition others to have them written) describing this flaw and how to correct it. Eliminating such a flaw or providing means of averting it would greatly aid LW and the community in general.
Unfortunately that is not currently possible for many reasons, including some large ones I can’t talk about and that I can’t talk about why I can’t talk about. I can’t see any way that it would become possible in the next few years either. I find this stressful; it’s why I make token attempts to communicate in extremely abstract or indirect ways with Less Wrong, despite the apparent fruitlessness. But there’s really nothing for it.
Unrelated public announcement: People who go back and downvote every comment someone’s made, please, stop doing that. It’s a clever way to pull information cascades in your direction but it is clearly an abuse of the content filtering system and highly dishonorable. If you truly must use such tactics, downvoting a few of your enemy’s top level posts is much less evil; your enemy loses the karma and takes the hint without your severely biasing the public perception of your enemy’s standard discourse. Please.
(I just lost 150 karma points in a few minutes and that’ll probably continue for awhile. This happens a lot.)
Why can’t you talk about why you can’t talk about them?
I’m not a big fan of the appeal to secret reasons, so I think I’m going to have pull out of this discussion. I will note, however, that you personally seem to be involved in more misunderstandings than the average LW poster, so while it’s certainly possible that your secret reasons are true and valid and Eliezer just sucks at reading or whatever, you may want to clarify certain elements of your own communication as well.
I unfortunately predict that “going more meta” will not be strongly received here.
I’m sorry to hear that you’re up against something so difficult, and I hope you find a way out.
Thank you… I think I just need to be more meta. Meta never fails.
Are we still talking about improving general reading comprehension? What could possibly be dangerous about that?
To save some time and clarify, this was option 3: an analysis that purports to say whether or not Friendly AI is “feasible”.
(Some people, including me, would really like it if a competent and FAI-focused uber-rationalist non-profit existed. I know people who will soon have enough momentum to make this happen. I am significantly more familiar with the specifics of FAI (and of hardcore SingInst-style rationality) than many of those people and almost anyone else in the world, so it’d be necessary that I put a lot of hours into working with those who are higher status than me and better at getting things done but less familiar with technical Friendliness. But I have many other things I could be doing. Hence the question.)
Does “FAI-focused” mean what I called code first? What are your thoughts on that post and its followup? What is this new non-profit planning to do differently from SIAI and why? What are the other things that you could be doing?
Incomplete response:
Jah. Well, at least determining whether or not “code first” is even reasonable, yeah, which is a difficult question in itself and only partially tied in with making direct progress on FAI.
You seem to have missed Oracle AI? (Eliezer’s dismissal of it isn’t particularly meaningful.) I agree with your concerns. This is why the main focus would at least initially be determining whether or not “code first” is a plausible approach (difficulty-wise and safety-wise). The value of information on that question is incredibly high and as you’ve pointed out it has not been sufficiently researched.
Basically everything. SingInst is focused on funding a large research program and gaining the prestige necessary to influence (academic) culture and academic and political policy. They’re not currently doing any research on Friendly AI, and their political situation is such that I don’t expect them to be able to do so effectively for a while, if ever. I will not clarify this. (Actually their research associates are working on FAI-related things, but SingInst doesn’t pay them to do that.)
Learning, mostly. Working with an unnamed group of x-risk-cognizant people that LW hasn’t heard of, in a way unrelated to their setting up a non-profit.
My understanding is that SIAI recently tried to set up a new in-house research team to do preliminary research into FAI (i.e., not try to build an FAI yet, but just do whatever research that might be eventually helpful to that project). This effort didn’t get off the ground, but my understanding again is that it was because the researchers they tried to recruit had various reasons for not joining SIAI at this time. I was one of those they tried to recruit, and while I don’t know what the others’ reasons were, mine were mostly personal and not related to politics.
You must also know all this, since you were involved in this effort. So I’m confused why you say SIAI won’t be doing effective research on FAI due to its “political situation”. Did the others not join SIAI because they thought SIAI was in a bad political situation? (This seems unlikely to me.) Or are you referring to the overall lack of qualified, recruitable researchers as a “political situation”? If you are, why do you think this new organization would be able to do better?
(Or did you perhaps not learn the full story, and thought SIAI stopped this effort for political reasons?)
The answer to your question isn’t among your list of possible answers. The recent effort to start an in-house research team was a good attempt and didn’t fail for political reasons. I am speaking of other things. However I want to take a few weeks off from discussion of such topics; I seem to have given off the entirely wrong impression and would prefer to start such discussion anew in a better context, e.g. one that better emphasizes cooperation and tentativeness rather than reactionary competition. My apologies.
If you can’t say anything, don’t say anything.
It’s a good heuristic, but can be very psychologically difficult. E.g. if you think that not even trying to communicate will be seen as unjustified in retrospect even if people should know that there was no obvious way for you to communicate. This has happened enough to me that the thought of just giving up on communication is highly aversive; my fear of being blamed for not preventing others to take unjustified actions (that will cause me, them, and the universe counterfactually-needless pain) is too great. But whatever, I’m starting to get over it.
Like, I remember a pigeon dying… people dying… a girl who starved herself… a girl who cut herself… a girl who wanted to commit suicide… just, trust me, there are reasons that I’m afraid. I could talk about those reasons but I’d rather not. It’s just, if you don’t even make a token gesture it’s like you don’t even care at all, and it’s easier to be unjustified in a way that can be made to look sorta like caring than in a way that looks like thoughtlessness or carelessness.
(ETA: A lot of the time when people give me or others advice I mentally translate it to “The solution is simple, just shut up and be evil.”.)
I have no particular attachments to SIAI and would love to see a more effective Singularitarian organization formed if that were possible. I’m just having genuine trouble understanding why you think this new proposed organization will be able to do more effective FAI research. Perhaps you could use these few weeks off to ask some trusted advisors how to better communicate this point. (I understand you have sensitive information that you can’t reveal, but I’m guessing that you can do better even within that constraint.)
This is exactly the strategy I’ve decided to enact, e.g. talking to Anna. Thanks for being… gentle, I suppose? I’ve been getting a lot of flak lately, it’s nice to get some non-insulting advice sometimes. :)
(Somehow I completely failed to communicate the tentativeness of the ideas I was throwing around; in my head I was giving it about a 3% chance that I’d actually work on helping build an organization but I seem to have given off an impression of about 30%. I think this caused everyone’s brains to enter politics mode, which is not a good mode for brains to be in.)
It’s rather strange how the SIAI is secretive. The military projects are secretive, the commercial projects are secretive alas: so few value transparency. An open project would surely do better, through being more obviously trustworthy and accountable, being better able to use talent across the internet, etc. I figure if the SIAI persists in not getting to grips with this issue, some other organisation will.
Could you tell us about them?