but he gives no evidence for this assertion. Calculating the decimals of pi may be a fairly simple mathematical operation that doesn’t have any need for superintelligence, and thus may be a really unlikely goal for a superintelligence—so that if you tried to build a superintelligence with this goal and connected it to the real world, it would very likely get its initial goal subverted and wind up pursuing some different, less idiotic goal.
Yes, it is fairly simple—a line of code. But in the real world, even humans who don’t have pi mentioned anywhere in their utility function can happily spend their lives working on mathematics—like pi. Pi is endlessly interesting: finding sequences in it (or humorous ones), proving properties like transcendentalness (or dare I say, normality?), coming up with novel algorithms and proving convergence, golfing short pi-generating programs, testing your routines, building custom supercomputers to calculate it—and think of how many scientific fields you need to build supercomputers!, depicting it as a graphic (entailing the entire field of data visualization, since what property do you want to see?), devising heuristic algorithms (entails much of statistics, since you might want optimal procedures for testing your heuristic pi-generating algorithms on subsequences of pi), writing books on all this, collaborating on all of the above, and silliness like Pi Day… I don’t know how one could more conclusively prove that pi is a perfectly doable obsession, given that this isn’t even plausible argumentation, it’s just pointing out facts about existing humans.
To summarize: http://en.wikipedia.org/wiki/Pi is really long. If you want to try to make an intuition pump argument-from-incredulity - ’oh surely an AI or superintelligence would get bored!′ - please pick something else, because pi is a horrible example.
“There are no uninteresting things, there are only uninterested people.”
If you want to try to make an intuition pump argument-from-incredulity - ‘oh surely an AI or superintelligence would get bored!’ - please pick something else, because pi is a horrible example.
FWIW, I don’t think that’s what Ben was doing. It seems more like a straw-man characterisation.
Alright, I have no idea what you’ve been talking about in any of your replies and as far as I can tell, at no point have I been unclear or mischaracterized Goertzel or Bostrom, so I’m bowing out.
You’re right, but isn’t this a needless distraction from the more important point, i.e. that it doesn’t matter whether we humans find interesting or valueable what the (unfriendly-)AI does?
I dunno, I think this is a pretty entertaining instance of anthropomorphizing + generalizing from oneself. At least in the future, I’ll be able to say things like “for example, Goertzel—a genuine AI researcher who has produced stuff—actually thinks that an intelligent AI can’t be designed to have an all-consuming interest in something like pi, despite all the real-world humans who are obsessed with pi!”
Ben:
Yes, it is fairly simple—a line of code. But in the real world, even humans who don’t have pi mentioned anywhere in their utility function can happily spend their lives working on mathematics—like pi. Pi is endlessly interesting: finding sequences in it (or humorous ones), proving properties like transcendentalness (or dare I say, normality?), coming up with novel algorithms and proving convergence, golfing short pi-generating programs, testing your routines, building custom supercomputers to calculate it—and think of how many scientific fields you need to build supercomputers!, depicting it as a graphic (entailing the entire field of data visualization, since what property do you want to see?), devising heuristic algorithms (entails much of statistics, since you might want optimal procedures for testing your heuristic pi-generating algorithms on subsequences of pi), writing books on all this, collaborating on all of the above, and silliness like Pi Day… I don’t know how one could more conclusively prove that pi is a perfectly doable obsession, given that this isn’t even plausible argumentation, it’s just pointing out facts about existing humans.
To summarize: http://en.wikipedia.org/wiki/Pi is really long. If you want to try to make an intuition pump argument-from-incredulity - ’oh surely an AI or superintelligence would get bored!′ - please pick something else, because pi is a horrible example.
FWIW, I don’t think that’s what Ben was doing. It seems more like a straw-man characterisation.
I agree it’s a strawman, but I think that’s exactly what Ben is doing because that is what he wrote.
Well, not the actual bit inside quotation marks. That was made up—and not a real quotation. He didn’t mention boredom either.
It’s not a real quotation? I seem to see it in Bostrom’s paper...
What—this one? Which quotation did you think I was talking about?
Alright, I have no idea what you’ve been talking about in any of your replies and as far as I can tell, at no point have I been unclear or mischaracterized Goertzel or Bostrom, so I’m bowing out.
You’re right, but isn’t this a needless distraction from the more important point, i.e. that it doesn’t matter whether we humans find interesting or valueable what the (unfriendly-)AI does?
I dunno, I think this is a pretty entertaining instance of anthropomorphizing + generalizing from oneself. At least in the future, I’ll be able to say things like “for example, Goertzel—a genuine AI researcher who has produced stuff—actually thinks that an intelligent AI can’t be designed to have an all-consuming interest in something like pi, despite all the real-world humans who are obsessed with pi!”