I think this may have been given a down vote because to some a Singularity seems settled and rational. I take it to be the idea that eventually—inevitably—machines will become smart enough to improve themselves, that once they become just a bit smarter than human programmers and with the capacity of self-modification, that it will be a runaway phenomenon.
And that’s why FAI is such a big deal: precisely because it’s “far into the fog of future” in the sense that we don’t have any control on the outcome (unless we align its values very precisely with our own).
But here’s the rub—for me, anyway—might it be rational to take your own reasoning as suspect about such ‘foggy’ matters? That is, even if you are doing your best to reason rationally, might it be best to assume that your potential knowledge is reduced to very little and the remainder filled in by cognitive bias?
Okay, maybe this is a better approach—since questions like Singularity&FAI are potentially so important—can anyone point me to a resource that convinced them of its reality?
I think this may have been given a down vote because to some a Singularity seems settled and rational. I take it to be the idea that eventually—inevitably—machines will become smart enough to improve themselves, that once they become just a bit smarter than human programmers and with the capacity of self-modification, that it will be a runaway phenomenon.
And that’s why FAI is such a big deal: precisely because it’s “far into the fog of future” in the sense that we don’t have any control on the outcome (unless we align its values very precisely with our own).
But here’s the rub—for me, anyway—might it be rational to take your own reasoning as suspect about such ‘foggy’ matters? That is, even if you are doing your best to reason rationally, might it be best to assume that your potential knowledge is reduced to very little and the remainder filled in by cognitive bias?
Okay, maybe this is a better approach—since questions like Singularity&FAI are potentially so important—can anyone point me to a resource that convinced them of its reality?
I would add, a resource that does not rely on the teachings of the local prophet, since most of us have already read those.