It is an open question whether our values and our lives will behave more like you have described or not.
For a lot of people, the desire to conform and not to be too weird by current human standards might make them converge over time. These people will live in highly customized utopias that suit their political and moral views, e.g. Christians in a mini-world where everyone has had their mind altered so that they can’t doubt God, can’t commit any sin, etc. E.g. ordinary modern semi-hedonists who live in something like the Culture. (Like pi as a number we use for engineering: the digits after the 100th convey almost no new information)
For others, boredom and curiosity will push them out into new territory. But the nature of both of these emotions is to incorporate environmental noise into one’s prefs. They’ll explore new forms of existence, new bodies, new emotions, etc, which will make them recursively weirder. These people will behave like the co-ordinates of a chaotic system in phase space: they will display very very high sensitivity to initial conditions, chance events “oh look, a flock of birds. I wonder what it would be like to exist as a swarm intelligence. I know, I’ll try it”.
The only group of people who I can see behaving the way you want are scientists. We have an abstract desire to understand how the world works. We will alter ourselves to become more intelligent in order to do so, and we have no idea what we will discover along the way. We are surely in for surprises as big as the discovery of evolution and quantum mechanics. Each new level of intelligence and discovery will be truly new, but hopefully the nature of truth is an abstract universal invariant that doesn’t depend upon the details of the path you take to get to it.
In essence, scientists are the only ones for whom long term optimization of our world has the kind of unbounded value that singularitarians want. Ordinary people will only get a limited amount of value out of a positive singularity. Thus their apathy about it is understandable.
@Vladimir:
It is an open question whether our values and our lives will behave more like you have described or not.
For a lot of people, the desire to conform and not to be too weird by current human standards might make them converge over time. These people will live in highly customized utopias that suit their political and moral views, e.g. Christians in a mini-world where everyone has had their mind altered so that they can’t doubt God, can’t commit any sin, etc. E.g. ordinary modern semi-hedonists who live in something like the Culture. (Like pi as a number we use for engineering: the digits after the 100th convey almost no new information)
For others, boredom and curiosity will push them out into new territory. But the nature of both of these emotions is to incorporate environmental noise into one’s prefs. They’ll explore new forms of existence, new bodies, new emotions, etc, which will make them recursively weirder. These people will behave like the co-ordinates of a chaotic system in phase space: they will display very very high sensitivity to initial conditions, chance events “oh look, a flock of birds. I wonder what it would be like to exist as a swarm intelligence. I know, I’ll try it”.
The only group of people who I can see behaving the way you want are scientists. We have an abstract desire to understand how the world works. We will alter ourselves to become more intelligent in order to do so, and we have no idea what we will discover along the way. We are surely in for surprises as big as the discovery of evolution and quantum mechanics. Each new level of intelligence and discovery will be truly new, but hopefully the nature of truth is an abstract universal invariant that doesn’t depend upon the details of the path you take to get to it.
In essence, scientists are the only ones for whom long term optimization of our world has the kind of unbounded value that singularitarians want. Ordinary people will only get a limited amount of value out of a positive singularity. Thus their apathy about it is understandable.