I was not sure whether to downvote this post for its epistemic value or upvote for instrumental (stimulating good discussion).
I ended up downvoting, I think this forum deserves better epistemic quality (I paused top-posting myself for this reason). I also donated to SIAI, because its value was once again validated to me by the discussion (though I have some reservations about apparent eccentricity of the SIAI folks, which is understandable (dropping out of high school is to me evidence of high rationality) but couterproductive (not having enough accepted academics involved). I mention this because it came up in the discussion and is definitely part of the subtext.
At to the concrete points of the post, I covered the part of it about the FAI vs AGI timeline here
The other part
Why is it that people like Vernor Vinge, Charles Stross or Ray Kurzweil are not running amok using all their influence to convince people of the risks ahead, or at least give all they have to the SIAI?
Is simply uninformed, and shows lack of diligence, which is the main reason I feel the post is not up to par and hope the clearly intelligent OP does some more homework and keeps contributing to the site.
Vinge has written about bad Singularity scenarios (his Singularity paper and sci-fi).
Stross has written about bad Singularity scenarios, at least in Accelerando (spoiler: humanity survives but only because AIs did not care about their resources at that point in time)
Kurzweil has written about the possibility of bad scenarios (CIO article in discussion below)
I’ll add one more, and to me rather damning: Peter Norvig, who wrote the (most widely used) book on AI and is head of research at Google is on the front page of SIAI (video clip), saying that as scientist we cannot ignore negative possibilities of AGI.
dropping out of high school is to me evidence of high rationality
Are you talking about me? I believe I’m the only person that could sorta kinda be affiliated with the Singularity Institute who has dropped out of high school, and I’m a lowly volunteer, not at all representative of the average credentials of the people who come through SIAI. Eliezer demonstrated his superior rationality to me by never going to high school in the first place. Damn him.
I finished high school early (16) by American standards, with college credit. By the more sane standards of Soviet education 16 is, well, standard (and you learn a lot more).
I was not sure whether to downvote this post for its epistemic value or upvote for instrumental (stimulating good discussion).
I ended up downvoting, I think this forum deserves better epistemic quality (I paused top-posting myself for this reason). I also donated to SIAI, because its value was once again validated to me by the discussion (though I have some reservations about apparent eccentricity of the SIAI folks, which is understandable (dropping out of high school is to me evidence of high rationality) but couterproductive (not having enough accepted academics involved). I mention this because it came up in the discussion and is definitely part of the subtext.
At to the concrete points of the post, I covered the part of it about the FAI vs AGI timeline here
The other part
Is simply uninformed, and shows lack of diligence, which is the main reason I feel the post is not up to par and hope the clearly intelligent OP does some more homework and keeps contributing to the site.
Vinge has written about bad Singularity scenarios (his Singularity paper and sci-fi).
Stross has written about bad Singularity scenarios, at least in Accelerando (spoiler: humanity survives but only because AIs did not care about their resources at that point in time)
Kurzweil has written about the possibility of bad scenarios (CIO article in discussion below)
I’ll add one more, and to me rather damning: Peter Norvig, who wrote the (most widely used) book on AI and is head of research at Google is on the front page of SIAI (video clip), saying that as scientist we cannot ignore negative possibilities of AGI.
Are you talking about me? I believe I’m the only person that could sorta kinda be affiliated with the Singularity Institute who has dropped out of high school, and I’m a lowly volunteer, not at all representative of the average credentials of the people who come through SIAI. Eliezer demonstrated his superior rationality to me by never going to high school in the first place. Damn him.
I dropped out of high school… to go to college early.
I finished high school early (16) by American standards, with college credit. By the more sane standards of Soviet education 16 is, well, standard (and you learn a lot more).
talking about this comment.
Here are a few comments where I advance on that particular point:
Comment 1
Comment 2
Comment 3
Comment 4