I’m not that confident, especially as I have a sneaking suspicion that something really weird is going on cosmologically speaking, but isn’t it the default assumption ’round these parts?
I’m not that confident, especially as I have a sneaking suspicion that something really weird is going on cosmologically speaking, but isn’t it the default assumption ’round these parts?
The default assumption is that there are (or will be) many other FAIs that light from our world-history will directly interact with? I didn’t know that. It’s certainly not mine. I thought it was more likely that we were for practical purposes alone. If you’ll pardon the shorthand reasoning:
Fermilike considerations… epically unlikely that life emerges all the way to superintelligence takeoff
Anthropics and self indication and suchforth… most EBs where one superintelligence emerges will not be branches in which more than one universe emerges.
There are heaps of other superintelligences, with high probability all the other ones are in parts of the (broadly used) Universe that are causally inaccessible.
The physics is complicated, often speculative (by folks like Tegmark) and beyond me but it all adds up to “we’re probably effectively alone in the universe as we see it”.
So you think there are other FAIs out there that our civilization would encounter if we got that far? How much does this depend on probability that we are in a simulation or under the benevolent (or otherwise) control of a powerful agent and how likely would you consider it to be conditional on us not being simulated/overseen?
how likely would you consider it to be conditional on us not being simulated/overseen?
So it’s possible that spacetime is infinitely dense and if you’re a superintelligence there’s no reason to expand. Dunno how likely that is, though blackholes do creep me out. Abiogenesis really doesn’t seem all that impossible, and anyway I think anthropic explanations are fundamentally confused. If your AI never expands then it can’t get precise info about its past, but maybe there are non-physical computational ways to do that, so the costs might not be worth the benefits. It seems like I might’ve been wrong in that LessWrong folk migh prefer anthropic solutons to Fermi, but I’m not sure how much evidence that is, especially as anthropics is confusing and possibly confused. So yeah… maybe 25% or so, but that’s only factoring in some structural uncertainty. Meh.
’Course, my primary hypothesis is that we are being overseen, and brains sometimes have trouble reasoning about hypothetical scenarios which aren’t already the default expectation. It’s at times like this when advanced rationality skills would be helpful.
Fermilike considerations… epically unlikely that life emerges all the way to superintelligence takeoff
I don’t follow. Do you think intelligences would loudly announce their existence over a long enough time period such that we would know about it? It always struck me as more likely that AGIs were quiet than that they didn’t exist. Remember, all those stars you see at night don’t necessarily exist; could just as easily be an illusion. All it’d take is for one superintelligence to show up somewhere and decide that we weren’t worth killing but that we shouldn’t get to see what’s actually going on as it gobbles all the unoccupied planets. There are various reasons it would want to do this. [ETA: The alternative, that abiogenesis is really difficult, strikes me as unlikely, and I have a very strong skepticism of anthropic “explanations”.]
There are heaps of other superintelligences, with high probability all the other ones are in parts of the (broadly used) Universe that are causally inaccessible
Hm, this might be a difference of perspective; I’m not very confident in the simulation argument as it’s usually put forth. (I tried to explain some of my reasons elsewhere in this thread.)
So you think there are other FAIs out there that our civilization would encounter if we got that far? How much does this depend on probability that we are in a simulation or under the benevolent (or otherwise) control of a powerful agent and how likely would you consider it to be conditional on us not being simulated/overseen?
(They don’t have to be Friendly, they just have to be willing to trade.) I don’t have a strong opinion either way. If we’re being overseen then it seems true by definition that we’ll run into other AGIs if we build an FAI, so I was focusing on the scenario where we’re not being overseen/simulated/fucked-with. In such a scenario I don’t know what probability to put on it… I’ll think about it more.
If we’re being overseen then it seems true by definition that we’ll run into other AGIs if we build an FAI
This seems likely but is not true by definition. In fact if I were designing and overseer I can see reasons why I may prefer to design one that keeps itself hidden except where intervention is required. Such an overseer, upon detecting that the overseen have created an AI with an acceptable goal system, may actively destroy all evidence of its existence.
True, mea culpa. I swear, there’s something about the words “by definition” that makes you misuse them even if you’re already aware of how often they’re misused. I almost never say “by definition” and yet it still screwed me over.
The alternative, that abiogenesis is really difficult, strikes me as unlikely, and I have a very strong skepticism of anthropic “explanations”.
I keep running into people who think anthropic reasoning doesn’t explain anything, or that have it entirely backwards. One prominent physicist whose name eludes me commented in an editorial published in physics today that anthropic reasoning was worthless unless the life-compatible section of the probability distribution of universal laws was especially likely. This so utterly misses the point that he clearly didn’t understand the basic argument.
I’ve never encountered anyone who’s willing to admit to buying anything stronger than the weak anthropic principle, which seems utterly obviously true:
1) If the universe didn’t enable the formation of sapient life, it wouldn’t exist (edited to clarify: sapient life, not the universe). If the universe made the formation of such life fantastically unlikely in any one location but the extent of the universe is larger than the reciprocal of that probability density, it would likely exist.
2) Our existence thus doesn’t indicate much about the general hospitability of the rules of the universe to the formation of sapient life, because the universe is awfully large, possibly infinite.
3) In the event that the rules of the universe that we observe are consequences of more fundamental laws, and those fundamental laws are quantum mechanical in nature so that multiple variants get a nonzero component, then the probability of life forming in this universe is taken as the OR among all of those variants.
There are heaps of other superintelligences, with high probability all the other ones are in parts of the (broadly used) Universe that are causally inaccessible
Hm, this might be a difference of perspective; I’m not very confident in the simulation argument as it’s usually put forth. (I tried to explain some of my reasons elsewhere in this thread.)
Is there a typo in there? The simulation argument doesn’t seem to fit.
Sorry, my sentence was unclear; “it” was referencing the belief that at least one intelligence besides us has already shown up or will show up somewhere in the universe at some point. It seems to me that most people, including most people on LessWrong, think this is likely.
I’m not that confident, especially as I have a sneaking suspicion that something really weird is going on cosmologically speaking, but isn’t it the default assumption ’round these parts?
The default assumption is that there are (or will be) many other FAIs that light from our world-history will directly interact with? I didn’t know that. It’s certainly not mine. I thought it was more likely that we were for practical purposes alone. If you’ll pardon the shorthand reasoning:
Fermilike considerations… epically unlikely that life emerges all the way to superintelligence takeoff
Anthropics and self indication and suchforth… most EBs where one superintelligence emerges will not be branches in which more than one universe emerges.
There are heaps of other superintelligences, with high probability all the other ones are in parts of the (broadly used) Universe that are causally inaccessible.
The physics is complicated, often speculative (by folks like Tegmark) and beyond me but it all adds up to “we’re probably effectively alone in the universe as we see it”.
So you think there are other FAIs out there that our civilization would encounter if we got that far? How much does this depend on probability that we are in a simulation or under the benevolent (or otherwise) control of a powerful agent and how likely would you consider it to be conditional on us not being simulated/overseen?
So it’s possible that spacetime is infinitely dense and if you’re a superintelligence there’s no reason to expand. Dunno how likely that is, though blackholes do creep me out. Abiogenesis really doesn’t seem all that impossible, and anyway I think anthropic explanations are fundamentally confused. If your AI never expands then it can’t get precise info about its past, but maybe there are non-physical computational ways to do that, so the costs might not be worth the benefits. It seems like I might’ve been wrong in that LessWrong folk migh prefer anthropic solutons to Fermi, but I’m not sure how much evidence that is, especially as anthropics is confusing and possibly confused. So yeah… maybe 25% or so, but that’s only factoring in some structural uncertainty. Meh.
’Course, my primary hypothesis is that we are being overseen, and brains sometimes have trouble reasoning about hypothetical scenarios which aren’t already the default expectation. It’s at times like this when advanced rationality skills would be helpful.
I don’t follow. Do you think intelligences would loudly announce their existence over a long enough time period such that we would know about it? It always struck me as more likely that AGIs were quiet than that they didn’t exist. Remember, all those stars you see at night don’t necessarily exist; could just as easily be an illusion. All it’d take is for one superintelligence to show up somewhere and decide that we weren’t worth killing but that we shouldn’t get to see what’s actually going on as it gobbles all the unoccupied planets. There are various reasons it would want to do this. [ETA: The alternative, that abiogenesis is really difficult, strikes me as unlikely, and I have a very strong skepticism of anthropic “explanations”.]
Hm, this might be a difference of perspective; I’m not very confident in the simulation argument as it’s usually put forth. (I tried to explain some of my reasons elsewhere in this thread.)
(They don’t have to be Friendly, they just have to be willing to trade.) I don’t have a strong opinion either way. If we’re being overseen then it seems true by definition that we’ll run into other AGIs if we build an FAI, so I was focusing on the scenario where we’re not being overseen/simulated/fucked-with. In such a scenario I don’t know what probability to put on it… I’ll think about it more.
This seems likely but is not true by definition. In fact if I were designing and overseer I can see reasons why I may prefer to design one that keeps itself hidden except where intervention is required. Such an overseer, upon detecting that the overseen have created an AI with an acceptable goal system, may actively destroy all evidence of its existence.
True, mea culpa. I swear, there’s something about the words “by definition” that makes you misuse them even if you’re already aware of how often they’re misused. I almost never say “by definition” and yet it still screwed me over.
I keep running into people who think anthropic reasoning doesn’t explain anything, or that have it entirely backwards. One prominent physicist whose name eludes me commented in an editorial published in physics today that anthropic reasoning was worthless unless the life-compatible section of the probability distribution of universal laws was especially likely. This so utterly misses the point that he clearly didn’t understand the basic argument.
I’ve never encountered anyone who’s willing to admit to buying anything stronger than the weak anthropic principle, which seems utterly obviously true:
1) If the universe didn’t enable the formation of sapient life, it wouldn’t exist (edited to clarify: sapient life, not the universe). If the universe made the formation of such life fantastically unlikely in any one location but the extent of the universe is larger than the reciprocal of that probability density, it would likely exist.
2) Our existence thus doesn’t indicate much about the general hospitability of the rules of the universe to the formation of sapient life, because the universe is awfully large, possibly infinite.
3) In the event that the rules of the universe that we observe are consequences of more fundamental laws, and those fundamental laws are quantum mechanical in nature so that multiple variants get a nonzero component, then the probability of life forming in this universe is taken as the OR among all of those variants.
That’s really all there is to it...
Is there a typo in there? The simulation argument doesn’t seem to fit.
Oh, I was assuming… never mind, it’s probably not worth untangling.
Not as far as I can tell. What do you mean?
Sorry, my sentence was unclear; “it” was referencing the belief that at least one intelligence besides us has already shown up or will show up somewhere in the universe at some point. It seems to me that most people, including most people on LessWrong, think this is likely.