The ideas really started forming around the recent ‘public relations’ discussions.
If we want to change people’s minds, we should be advertising.
My guess is that “Harry Potter and the Methods of Rationality” is the best piece of publicity the SIAI has ever produced.
I think that the only way to top it would be a Singularity/FAI-themed computer game.
How about a turn-based strategy game where the object is to get deep enough into the singularity to upload yourself before a uFAI shows up and turns the universe into paper clips?
Maybe it would work, and maybe not, but I think that the demographic we want to reach is 4chan—teenage hackers. We need to tap into the “Dark Side” of the Cyberculture.
How about a turn-based strategy game where the object is to get deep enough into the singularity to upload yourself before a uFAI shows up and turns the universe into paper clips?
I don’t think that would be very helpful. Advocating rationality (even through Harry Potter fanfiction) helps because people are better at thinking about the future and existential risks when they care about and understand rationality. But spreading singularity memes as a kind of literary genre won’t do that. (With all due respect, your idea doesn’t even make sense: I don’t think “deep enough into the singularity” means anything with respect to what we actually talk about as the “singularity” here (successfully launching a Friendly singularity probably means the world is going to be remade in weeks or days or hours or minutes, and it probably means we’re through with having to manually save the world from any remaining threats), and if a uFAI wants to turn the universe into paperclips, then you’re screwed anyway, because the computer you just uploaded yourself into is part of the universe.)
Unfortunately, I don’t think we can get people excited about bringing about a Friendly singularity by speaking honestly about how it happens purely at the object level, because what actually needs to be done is tons of math (plus some outreach and maybe paper-writing and book-writing and eventually a lot of coding). Saving the world isn’t actually going to be an exciting ultimate showdown of ultimate destiny, and any marketing and publicity shouldn’t be setting people up for disappointment by portraying it as such… and it should also be making it clear that even if existential risk reduction were fun and exciting, it wouldn’t be something you do for yourself because it’s fun and exciting, and you don’t do it because you get to affiliate with smart/high-status people and/or become known as one yourself, and you don’t do it because you personally want to live forever and don’t care about the rest of the world, you do it because it’s the right thing to do no matter how little you personally get out of it.
So we don’t want to push the public further toward thinking of the singularity as a geek / sci-fi / power-fantasy / narcissistic thing (I realize some of those are automatic associations and pattern completions that people independently generate, but that’s to be resisted and refuted rather than embraced). Fiction that portrays rationality as virtuous (and transparent, as in the Rationalist Fanfiction Principle) and that portrays transhumanistic protagonists that people can identify with (or at least like) is good because it makes the right methods and values salient and sympathetic and exciting. Giving people a vision of a future where humanity has gotten its shit together as a thing-to-protect is good; anything that makes AI or the Singularity or even FAI seem too much like an end in itself will probably be detrimental, especially if it is portrayed anywhere near anthropomorphically enough for it to be a protagonist or antagonist in a video game.
Maybe it would work, and maybe not, but I think that the demographic we want to reach is 4chan—teenage hackers. We need to tap into the “Dark Side” of the Cyberculture.
Only if they can be lured to the Light Side. The *chans seem rather tribal and amoral (at least the /b/s and the surrounding culture; I know that’s not the entirety of the *chans, but they have the strongest influence in those circles). If the right marketing can turn them from apathetic tribalist sociopaths into altruistic globalist transhumanists, then that’s great, but I wouldn’t focus limited resources in that direction. Probably better to reach out to academia; at least that culture is merely inefficient rather than actively evil.
I don’t think that would be very helpful. [And here is why...]
I am impressed. A serious and thoughtful reply to a maybe serious, but definitely not thoughtful, suggestion. Thank you.
If the right marketing can turn them [the *chans] from apathetic tribalist sociopaths into altruistic globalist transhumanists, then that’s great, but I wouldn’t focus limited resources in that direction. Probably better to reach out to academia; at least that culture is merely inefficient rather than actively evil.
“Actively evil” is not “inherently evil”. The action currently is over on the evil side because the establishment is boring. Anti-establishment evil is currently more fun. But what happens if the establishment becomes evil and boring? Could happen on the way to a friendly singularity. Don’t rule any strategies out. Thwarting a nascent uFAI may be one of the steps we need to take along the path to FAI.
I am impressed. A serious and thoughtful reply to a maybe serious, but definitely not thoughtful, suggestion. Thank you.
Thank you for taking it well; sometimes I still get nervous about criticizing. :)
“Actively evil” is not “inherently evil”. The action currently is over on the evil side because the establishment is boring. Anti-establishment evil is currently more fun. But what happens if the establishment becomes evil and boring? Could happen on the way to a friendly singularity. Don’t rule any strategies out. Thwarting a nascent uFAI may be one of the steps we need to take along the path to FAI.
I’ve heard the /b/ / “Anonymous” culture described as Chaotic Neutral, which seems apt. My main concern is that waiting for the right thing to become fun for them to rebel against is not efficient. (Example: Anonymous’s movement against Scientology began not in any of the preceding years when Scientology was just as harmful as always, but only once they got an embarrassing video of Tom Cruise taken down from YouTube. “Project Chanology” began not as anything altruistic, but as a morally-neutral rebellion against what was perceived as anti-lulz. It did eventually grow into a larger movement including people who had never heard of “Anonymous” before, people who actually were in it to make the world a better place whether the process was funny or not. These people were often dismissed as “moralfags” by the 4chan old-timers.) Indeed they are not inherently evil, but when morality is not a strong consideration one way or the other, it’s too easy for evil to be more fun than good. I would not rely on them (or even expect them) to accomplish any long-term good when that’s not what they’re optimizing for.
(And there’s the usual “herding cats” problem — even if something would normally seem fun to them, they’re not going to be interested if they get the sense that someone is trying to use them.)
Maybe some useful goal that appeals to their sensibilities will eventually present itself, but for now, if we’re thinking about where to direct limited resources and time and attention, putting forth the 4chan crowd as a good target demographic seems like a privileged hypothesis. “Teenage hackers” are great (I was one!), but I’m not sure about reaching out to them once they’re already involved in 4chan-type cultures. There are probably better times and places to get smart young people interested.
Any ideas. For the SIAI it would probably be existential risks then UFAI later, in general it could be rationality or evolution or atheism or whatever.
What is the whole industry you speak of? Self-help, religion, marketing? And what additional advertising? I think that spreading the ideas is important as well, I”m just not sure what you are considering.
Advertising/marketing. Short of ashiest bus ads, I can’t think of anything that’s been done.
All I’m really suggesting is that we focus on mass persuasion in the way it has been proven to be most efficient. What that actually amounts to will depend on the target audience, and how much money is available, among other things.
Did you mean “atheist bus ads”? I actually find strict-universal-atheism to be irrational compared to agnosticism because of the SA and the importance of knowing the limits of certainty, but that’s unrelated and I digress.
I’ve long suspected that writing popular books on the subject would be an effective strategy for mass persuasion. Kurzweil has certainly had a history of some success there, although he also brings some negative publicity due to his association with dubious supplements and the expensive SingUniversity. It will be interesting to see how EY’s book turns out and is received.
I’m actually skeptical about how far rationality itself can go towards mass persuasion. Building a rational case is certainly important, but the content of your case is even more important (regardless of its rationality).
On that note I suspect that bridging a connection to the mainstream’s beliefs and values would go a ways towards increasing mass marketability. You have to consider not just the rationality of ideas, but the utility of ideas.
It would be interesting to analyze and compare how emphasizing the hope vs doom aspects of the message would effect popularity. SIAI at the moment appears focused on emphasizing doom and targeting a narrow market: a subset of technophile ‘rationalists’ or atheist intellectuals and wooing academia in particular.
I’m interested in how you’d target mainstream liberal christians or new agers, for example, or even just the intellectual agnostic/atheist mainstream—the types of people who buy books such as the End of Faith, Breaking the Spell, etc etc. Although a good portion of that latter demographic is probably already exposed to the Singularity is Near.
I’m not sure what I’d do, but I’m not a marketing expert either. (Though I am experimenting)
It would probably be possible to make a campaign that took advantage of UFAI in sci-fi. AI’s taking over the world isn’t a difficult concept to get across, so the ad would just need to persuade that it’s possible in reality, and there is a group working towards a solution.
I hope you haven’t forgotten our long drawn out discussion, as I do think that one is worthwhile.
AI’s taking over the world isn’t a difficult concept to get across
AIs taking over the world because they have implausibly human-like cognitive architectures and they hate us or resent us or desire higher status than us is an easy concept to get across. It is also, of course, wrong. An AI immediately taking apart the world to use its mass for something else because its goal system is nothing like ours and its utility function doesn’t even have a term for human values is more difficult; because of anthropomorphic bias, it will be much less salient to people, even if it is more probable.
They have the right conclusion (plausible AI takeover) for slightly wrong reasons. “hate [humans] or resent [humans] or desire higher status than [humans]” are slightly different values than ours (even if just like the values humans often have towards other groups)
So we can gradually nudge people closer to the truth a bit at a time by saying “Plus, it’s unlikely that they’ll value X, so even if they do something with the universe it will not have X”
But we don’t have to introduce them to the full truth immediately, as long as we don’t base any further arguments on falsehoods they believe.
If someone is convinced of the need for asteroid defense because asteroids could destroy a city, you aren’t obligated to tell them that larger asteroids could destroy all humanity when you’re asking for money. Even if you believe bigger asteroids to be more likely.
I don’t think it’s dark epistemology to avoid confusing people if they’ve already got the right idea.
So we can gradually nudge people closer to the truth a bit at a time by saying “Plus, it’s unlikely that they’ll value X, so even if they do something with the universe it will not have X”
Writing up high-quality arguments for your full position might be a better tool than “nudging people closer to the truth a bit at a time”. Correct ideas have a scholar appeal due to internal coherence, even if they need to overcome plenty of cached misconceptions, but making that case requires a certain critical mass of published material.
I do see value in that, but I’m thinking of a TV commercial or youtube video with a terminator style look and feel. Though possibly emphasizing that against real superintelligence, there would be no war.
I can’t immediately remember a way to simplify “the space of all possible values is huge and human like values are a tiny part of that” and I don’t think that would resonate at all.
I do see value in that, but I’m thinking of a TV commercial or youtube video with a terminator style look and feel.
A large portion of the world has already seen a Terminator flick, or the Matrix. The AI-is-evil-nonhuman-threat meme is already well established in the wild, to the point of characterture. The AI-is-an-innocent-child meme wasn’t as popular—AI is the only example I can think of and not many people saw it.
And even though the Terminator and the Matrix are far from realistic, they did at least get the general shape of the outcome correct—humans lose.
What would your message add over this in reach or content?
At this point the meme is almost oversaturated and it is difficult for people to take seriously. Did “The Day After Tommorrow” help or hinder the environmental movement?
This might not fit the terminator motif anymore, but:
That there are people working on a way to target AI development so it reliably looks more like R2D2, Johnny 5, Commander Data, Sonny, Marvin… ok that’s all I can think of but just for fun I’ll get these from wikipedia:
Gort, Bishop from aliens, almost everything from the jetsons, Transformers (autobots anyway), the Iron Giant, and KITT
And again we don’t have to explain that AI done right will be orders of magnitude more helpful than any of these.
It’s interesting that friendly-AI was so common in earlier decades and then this seemed to shift in the 90′s.
As for AI-positive advertisements, that somehow reminded me. . .
did you ever see that popular web-viral anti-banking video called Zeitgeist? In the sequel he seems to have realized that just being a critic wasn’t enough, so suddenly the 2nd part of Zeitgeist addendum turns into a startrek-ish utopia proposal out of nowhere. I forget the name, but it is basically some architect’s pseudo-singularity (AI solves all our problems and makes these beautiful new cities for us but isn’t really conscious or dangerous).
I went to a screening of that film in LA, and I was amazed at how entranced the audience seemed to be. The questions at the end were pretty funny too -
“so .. there won’t be any money? And the AI’s will build us whatever we want?”
“Yes”
“So, what if I want to turn all of Texas into my house?”
AIs taking over the world because they have implausibly human-like cognitive architectures and they hate us or resent us or desire higher status than us is an easy concept to get across. An AI immediately taking apart the world to use its mass for something else because its goal system is nothing like ours and its utility function doesn’t even have a term for human values is more difficult; because of anthropomorphic bias, it will be much less salient to people, even if it is more probable.
I actually come from that outside-LW viewpoint that finds the former scenario involving “human-like cognitive architectures” as vastly more probable than “AI immediately taking apart the world to use its mass for something else because its goal system is nothing like ours and its utility function doesn’t even have a term for human values”.
So it could be that your viewpoint is more likely, and the rest of us are suffering from “anthropomorphic bias”, but it also could be that anthropomorphic bias is in fact a self-fulfilling prophecy.
So it could be that your viewpoint is more likely, and the rest of us are suffering from “anthropomorphic bias”, but it also could be that anthropomorphic bias is in fact a self-fulfilling prophecy.
I don’t see how. We could get something like that if we get uploads before AGI, but that would really be more like an enhanced human taking over the world. Aside from that, where’s the self-fulfilling prophecy? If people expect AGIs to exhibit human-like emotions and primate status drives and go terribly wrong as a result, why does that increase the chance that the creators of the first powerful AGI will build human-like emotions and primate status drives into it?
Actual uploads are a far end point along a continuum of human-like cognitive architectures, and have the additional complexity of scanning technology which lags far behind electronics. You don’t need uploads for anthropomorphic AI—you just need to loosely reverse engineer the brain.
Also, “human-like cognitive architectures” is a wide spectrum that does not require human-like emotions or primate status drives—consider the example of Alexithymia.
Understanding human languages is a practical prerequisite for any AI to reach high levels of intelligence, and the implied anthropomorphic cognitive capacities required for true linguistic thinking heavily constrains the design space.
The self-fulfilling prophecy is that anthropomorphic AI will be both easier for us to create and more useful for us—so the bias is correct in a self-reinforcing manner.
Finally Prompted by this, but it would be too offtopic there
http://lesswrong.com/lw/2ot/somethings_wrong/
The ideas really started forming around the recent ‘public relations’ discussions.
If we want to change people’s minds, we should be advertising.
I do like long drawn out debates, but most of the time they don’t accomplish anything and even when they do, they’re a huge use of personal resources.
There is a whole industry centered around changing people’s minds effectively. They have expertise in this, and they do it way better than we do.
My guess is that “Harry Potter and the Methods of Rationality” is the best piece of publicity the SIAI has ever produced.
I think that the only way to top it would be a Singularity/FAI-themed computer game.
How about a turn-based strategy game where the object is to get deep enough into the singularity to upload yourself before a uFAI shows up and turns the universe into paper clips?
Maybe it would work, and maybe not, but I think that the demographic we want to reach is 4chan—teenage hackers. We need to tap into the “Dark Side” of the Cyberculture.
I don’t think that would be very helpful. Advocating rationality (even through Harry Potter fanfiction) helps because people are better at thinking about the future and existential risks when they care about and understand rationality. But spreading singularity memes as a kind of literary genre won’t do that. (With all due respect, your idea doesn’t even make sense: I don’t think “deep enough into the singularity” means anything with respect to what we actually talk about as the “singularity” here (successfully launching a Friendly singularity probably means the world is going to be remade in weeks or days or hours or minutes, and it probably means we’re through with having to manually save the world from any remaining threats), and if a uFAI wants to turn the universe into paperclips, then you’re screwed anyway, because the computer you just uploaded yourself into is part of the universe.)
Unfortunately, I don’t think we can get people excited about bringing about a Friendly singularity by speaking honestly about how it happens purely at the object level, because what actually needs to be done is tons of math (plus some outreach and maybe paper-writing and book-writing and eventually a lot of coding). Saving the world isn’t actually going to be an exciting ultimate showdown of ultimate destiny, and any marketing and publicity shouldn’t be setting people up for disappointment by portraying it as such… and it should also be making it clear that even if existential risk reduction were fun and exciting, it wouldn’t be something you do for yourself because it’s fun and exciting, and you don’t do it because you get to affiliate with smart/high-status people and/or become known as one yourself, and you don’t do it because you personally want to live forever and don’t care about the rest of the world, you do it because it’s the right thing to do no matter how little you personally get out of it.
So we don’t want to push the public further toward thinking of the singularity as a geek / sci-fi / power-fantasy / narcissistic thing (I realize some of those are automatic associations and pattern completions that people independently generate, but that’s to be resisted and refuted rather than embraced). Fiction that portrays rationality as virtuous (and transparent, as in the Rationalist Fanfiction Principle) and that portrays transhumanistic protagonists that people can identify with (or at least like) is good because it makes the right methods and values salient and sympathetic and exciting. Giving people a vision of a future where humanity has gotten its shit together as a thing-to-protect is good; anything that makes AI or the Singularity or even FAI seem too much like an end in itself will probably be detrimental, especially if it is portrayed anywhere near anthropomorphically enough for it to be a protagonist or antagonist in a video game.
Only if they can be lured to the Light Side. The *chans seem rather tribal and amoral (at least the /b/s and the surrounding culture; I know that’s not the entirety of the *chans, but they have the strongest influence in those circles). If the right marketing can turn them from apathetic tribalist sociopaths into altruistic globalist transhumanists, then that’s great, but I wouldn’t focus limited resources in that direction. Probably better to reach out to academia; at least that culture is merely inefficient rather than actively evil.
I am impressed. A serious and thoughtful reply to a maybe serious, but definitely not thoughtful, suggestion. Thank you.
“Actively evil” is not “inherently evil”. The action currently is over on the evil side because the establishment is boring. Anti-establishment evil is currently more fun. But what happens if the establishment becomes evil and boring? Could happen on the way to a friendly singularity. Don’t rule any strategies out. Thwarting a nascent uFAI may be one of the steps we need to take along the path to FAI.
Thank you for taking it well; sometimes I still get nervous about criticizing. :)
I’ve heard the /b/ / “Anonymous” culture described as Chaotic Neutral, which seems apt. My main concern is that waiting for the right thing to become fun for them to rebel against is not efficient. (Example: Anonymous’s movement against Scientology began not in any of the preceding years when Scientology was just as harmful as always, but only once they got an embarrassing video of Tom Cruise taken down from YouTube. “Project Chanology” began not as anything altruistic, but as a morally-neutral rebellion against what was perceived as anti-lulz. It did eventually grow into a larger movement including people who had never heard of “Anonymous” before, people who actually were in it to make the world a better place whether the process was funny or not. These people were often dismissed as “moralfags” by the 4chan old-timers.) Indeed they are not inherently evil, but when morality is not a strong consideration one way or the other, it’s too easy for evil to be more fun than good. I would not rely on them (or even expect them) to accomplish any long-term good when that’s not what they’re optimizing for.
(And there’s the usual “herding cats” problem — even if something would normally seem fun to them, they’re not going to be interested if they get the sense that someone is trying to use them.)
Maybe some useful goal that appeals to their sensibilities will eventually present itself, but for now, if we’re thinking about where to direct limited resources and time and attention, putting forth the 4chan crowd as a good target demographic seems like a privileged hypothesis. “Teenage hackers” are great (I was one!), but I’m not sure about reaching out to them once they’re already involved in 4chan-type cultures. There are probably better times and places to get smart young people interested.
What ideas? I’m pretty sure I find whatever you are talking about interesting and shiny, but I’m not quite sure what it even is.
Any ideas. For the SIAI it would probably be existential risks then UFAI later, in general it could be rationality or evolution or atheism or whatever.
What is the whole industry you speak of? Self-help, religion, marketing? And what additional advertising? I think that spreading the ideas is important as well, I”m just not sure what you are considering.
Advertising/marketing. Short of ashiest bus ads, I can’t think of anything that’s been done.
All I’m really suggesting is that we focus on mass persuasion in the way it has been proven to be most efficient. What that actually amounts to will depend on the target audience, and how much money is available, among other things.
Did you mean “atheist bus ads”? I actually find strict-universal-atheism to be irrational compared to agnosticism because of the SA and the importance of knowing the limits of certainty, but that’s unrelated and I digress.
I’ve long suspected that writing popular books on the subject would be an effective strategy for mass persuasion. Kurzweil has certainly had a history of some success there, although he also brings some negative publicity due to his association with dubious supplements and the expensive SingUniversity. It will be interesting to see how EY’s book turns out and is received.
I’m actually skeptical about how far rationality itself can go towards mass persuasion. Building a rational case is certainly important, but the content of your case is even more important (regardless of its rationality).
On that note I suspect that bridging a connection to the mainstream’s beliefs and values would go a ways towards increasing mass marketability. You have to consider not just the rationality of ideas, but the utility of ideas.
It would be interesting to analyze and compare how emphasizing the hope vs doom aspects of the message would effect popularity. SIAI at the moment appears focused on emphasizing doom and targeting a narrow market: a subset of technophile ‘rationalists’ or atheist intellectuals and wooing academia in particular.
I’m interested in how you’d target mainstream liberal christians or new agers, for example, or even just the intellectual agnostic/atheist mainstream—the types of people who buy books such as the End of Faith, Breaking the Spell, etc etc. Although a good portion of that latter demographic is probably already exposed to the Singularity is Near.
I’m not sure what I’d do, but I’m not a marketing expert either. (Though I am experimenting)
It would probably be possible to make a campaign that took advantage of UFAI in sci-fi. AI’s taking over the world isn’t a difficult concept to get across, so the ad would just need to persuade that it’s possible in reality, and there is a group working towards a solution.
I hope you haven’t forgotten our long drawn out discussion, as I do think that one is worthwhile.
AIs taking over the world because they have implausibly human-like cognitive architectures and they hate us or resent us or desire higher status than us is an easy concept to get across. It is also, of course, wrong. An AI immediately taking apart the world to use its mass for something else because its goal system is nothing like ours and its utility function doesn’t even have a term for human values is more difficult; because of anthropomorphic bias, it will be much less salient to people, even if it is more probable.
They have the right conclusion (plausible AI takeover) for slightly wrong reasons. “hate [humans] or resent [humans] or desire higher status than [humans]” are slightly different values than ours (even if just like the values humans often have towards other groups)
So we can gradually nudge people closer to the truth a bit at a time by saying “Plus, it’s unlikely that they’ll value X, so even if they do something with the universe it will not have X”
But we don’t have to introduce them to the full truth immediately, as long as we don’t base any further arguments on falsehoods they believe.
If someone is convinced of the need for asteroid defense because asteroids could destroy a city, you aren’t obligated to tell them that larger asteroids could destroy all humanity when you’re asking for money. Even if you believe bigger asteroids to be more likely.
I don’t think it’s dark epistemology to avoid confusing people if they’ve already got the right idea.
Writing up high-quality arguments for your full position might be a better tool than “nudging people closer to the truth a bit at a time”. Correct ideas have a scholar appeal due to internal coherence, even if they need to overcome plenty of cached misconceptions, but making that case requires a certain critical mass of published material.
I do see value in that, but I’m thinking of a TV commercial or youtube video with a terminator style look and feel. Though possibly emphasizing that against real superintelligence, there would be no war.
I can’t immediately remember a way to simplify “the space of all possible values is huge and human like values are a tiny part of that” and I don’t think that would resonate at all.
A large portion of the world has already seen a Terminator flick, or the Matrix. The AI-is-evil-nonhuman-threat meme is already well established in the wild, to the point of characterture. The AI-is-an-innocent-child meme wasn’t as popular—AI is the only example I can think of and not many people saw it.
And even though the Terminator and the Matrix are far from realistic, they did at least get the general shape of the outcome correct—humans lose.
What would your message add over this in reach or content?
At this point the meme is almost oversaturated and it is difficult for people to take seriously. Did “The Day After Tommorrow” help or hinder the environmental movement?
This might not fit the terminator motif anymore, but:
That there are people working on a way to target AI development so it reliably looks more like R2D2, Johnny 5, Commander Data, Sonny, Marvin… ok that’s all I can think of but just for fun I’ll get these from wikipedia:
Gort, Bishop from aliens, almost everything from the jetsons, Transformers (autobots anyway), the Iron Giant, and KITT
And again we don’t have to explain that AI done right will be orders of magnitude more helpful than any of these.
It’s interesting that friendly-AI was so common in earlier decades and then this seemed to shift in the 90′s.
As for AI-positive advertisements, that somehow reminded me. . .
did you ever see that popular web-viral anti-banking video called Zeitgeist? In the sequel he seems to have realized that just being a critic wasn’t enough, so suddenly the 2nd part of Zeitgeist addendum turns into a startrek-ish utopia proposal out of nowhere. I forget the name, but it is basically some architect’s pseudo-singularity (AI solves all our problems and makes these beautiful new cities for us but isn’t really conscious or dangerous).
I went to a screening of that film in LA, and I was amazed at how entranced the audience seemed to be. The questions at the end were pretty funny too -
“so .. there won’t be any money? And the AI’s will build us whatever we want?”
“Yes”
“So, what if I want to turn all of Texas into my house?”
. . .
You are thinking of Jacque Fresco.
I actually come from that outside-LW viewpoint that finds the former scenario involving “human-like cognitive architectures” as vastly more probable than “AI immediately taking apart the world to use its mass for something else because its goal system is nothing like ours and its utility function doesn’t even have a term for human values”.
So it could be that your viewpoint is more likely, and the rest of us are suffering from “anthropomorphic bias”, but it also could be that anthropomorphic bias is in fact a self-fulfilling prophecy.
I don’t see how. We could get something like that if we get uploads before AGI, but that would really be more like an enhanced human taking over the world. Aside from that, where’s the self-fulfilling prophecy? If people expect AGIs to exhibit human-like emotions and primate status drives and go terribly wrong as a result, why does that increase the chance that the creators of the first powerful AGI will build human-like emotions and primate status drives into it?
Actual uploads are a far end point along a continuum of human-like cognitive architectures, and have the additional complexity of scanning technology which lags far behind electronics. You don’t need uploads for anthropomorphic AI—you just need to loosely reverse engineer the brain.
Also, “human-like cognitive architectures” is a wide spectrum that does not require human-like emotions or primate status drives—consider the example of Alexithymia.
Understanding human languages is a practical prerequisite for any AI to reach high levels of intelligence, and the implied anthropomorphic cognitive capacities required for true linguistic thinking heavily constrains the design space.
The self-fulfilling prophecy is that anthropomorphic AI will be both easier for us to create and more useful for us—so the bias is correct in a self-reinforcing manner.