This line of reasoning is hardly limited to CEV. I’m reminded of Bishop Wright’s apocryphal sermon about how we’ve pretty much discovered everything there is to discover.
Sure, if we progress far enough, fast enough, that all the meaningful problems I can conceive of are solved and all the meaningful goals I can imagine are reached, then there will only be two kinds of people: people solving problems I can’t conceive of in order to achieve goals I can’t imagine, and people living without meaningful goals and problems. We have the latter group today; I suspect we’ll have them tomorrow as well.
The possibility of their continued existence—even the possibility that everyone will be in that category—doesn’t strike me as a good enough reason to avoid such progress.
I’m also tempted to point out that there’s something inherently inconsistent about a future where the absence of meaningful problems to solve is a meaningful problem, although I suspect that’s just playing with words.
I’m also tempted to point out that there’s something inherently inconsistent about a future where the absence of meaningful problems to solve is a meaningful problem, although I suspect that’s just playing with words.
I don’t think that’s just playing with words. If we’ve solved all the problems, then we’ve solved that problem. We shouldn’t assume a priori that solving that problem is impossible.
I agree with you that we shouldn’t assume that finding meaningful activities for people to engage in as we progress is impossible. Not least of which because I think it is possible.
Actually, I’d say something stronger: I think right now we suck as a species at understanding what sorts of activities are meaningful and how to build a social infrastructure that creates such activities, and that we are suffering for the lack of it (and have done for millenia), and that we are just starting to develop tools with which to engage with this problem efficiently. In a few generations we might really see some progress in this area.
Nevertheless, I suspect that an argument of the form “lack of meaningful activity due to the solving of all problems is a logical contradiction, because such a lack of meaningful activity would then be an unsolved problem” is just playing with words, because the supposed contradiction is due entirely to the fact that the word “problem” means subtly different things in its two uses in that sentence.
I don’t mean anything deep by it, just that for example a system might be able to optimize our environment to .99 human-optimal (which is pretty well approximated by the phrase “solving all problems”) and thereby create, say, a pervasive and crippling sense of ennui that it can’t yet resolve (which would constitute a “problem”). There’s no contradiction in that scenario; the illusion of contradiction is created entirely by the sloppiness of language.
I don’t think I follow; if the environment is .99 human-optimal, then that remaining .01 gap implies that there are some problems remain to be solved, however few or minor, right?
It might simply be impossible to solve all problems, because of conflicting dependencies.
Yes, I agree that the remaining .01 gap represents problems that remain to be solved, which implies that “solving all problems” doesn’t literally apply to that scenario. If you’re suggesting that therefore such a scenario isn’t well-enough approximated by the phrase “solving all problems” to justify the phrase’s use, we have different understandings of the level of justification required.
The problem is not that all problems might be solved at some point but that as long as we don’t turn ourselves into something similarly capable as the CEV process, then there exists an oracle that we could ask if we wanted to. The existence of such an oracle is that which is diminishing the value of research and discovery.
I agree that the existence of such an oracle closes off certain avenues of research and discovery.
I’m not sure that keeping those avenues of research and discovery open is particularly important, but I agree that it might be.
If it turns out that the availability such an oracle closing off certain avenues is an important human problem, it seems to follow that any system capable of and motivated to solve human problems will ensure that no such oracle is available.
The existence of such an oracle is that which is diminishing the value of research and discovery
You seem to be saying that research and discovery has some intrinsic value, in addition to the benefits of actually discovering things and understanding them. If so, what is this value ?
The only answer I can think of is something like, “learning about new avenues of research that the oracle had not yet explored”, but I’m not sure whether that makes sense or not—since the perfect oracle would explore every avenue of research, and an imperfect oracle would strive toward perfection (as long as the oracle is rational).
Well, the process of research and discovery can itself be enjoyable. That said, I don’t feel that there is a need to hold onto our current enjoyable activities if a CEV can create novel superior ways for us to have fun.
I would posit that divergent behaviors and approaches to the norms will still occur, despite the existence of such an oracle just for the sake of imagination, exploration, and the enjoyment of the process itself. Such oracle would also be aware of the existence of unknown future factors, and the benefits of diverse approaches to problems in the face of factors with unknown long term benefits and viability until certain processes has been executed. As you said, such an oracle would then try to explore every avenue of research, while still focusing on the ones deemed most likely to be fruitful. Such oracle should also be good on self-reflection, and able to question its own approaches and the various perspectives it is able to subsume. After all, isn’t self introspection and self reflections part of how one improve themselves?
Then there’s the Fun theory sequence that DSimon have posted about.
This line of reasoning is hardly limited to CEV. I’m reminded of Bishop Wright’s apocryphal sermon about how we’ve pretty much discovered everything there is to discover.
Sure, if we progress far enough, fast enough, that all the meaningful problems I can conceive of are solved and all the meaningful goals I can imagine are reached, then there will only be two kinds of people: people solving problems I can’t conceive of in order to achieve goals I can’t imagine, and people living without meaningful goals and problems. We have the latter group today; I suspect we’ll have them tomorrow as well.
The possibility of their continued existence—even the possibility that everyone will be in that category—doesn’t strike me as a good enough reason to avoid such progress.
I’m also tempted to point out that there’s something inherently inconsistent about a future where the absence of meaningful problems to solve is a meaningful problem, although I suspect that’s just playing with words.
I don’t think that’s just playing with words. If we’ve solved all the problems, then we’ve solved that problem. We shouldn’t assume a priori that solving that problem is impossible.
See also: fun theory.
I agree with you that we shouldn’t assume that finding meaningful activities for people to engage in as we progress is impossible. Not least of which because I think it is possible.
Actually, I’d say something stronger: I think right now we suck as a species at understanding what sorts of activities are meaningful and how to build a social infrastructure that creates such activities, and that we are suffering for the lack of it (and have done for millenia), and that we are just starting to develop tools with which to engage with this problem efficiently. In a few generations we might really see some progress in this area.
Nevertheless, I suspect that an argument of the form “lack of meaningful activity due to the solving of all problems is a logical contradiction, because such a lack of meaningful activity would then be an unsolved problem” is just playing with words, because the supposed contradiction is due entirely to the fact that the word “problem” means subtly different things in its two uses in that sentence.
Can you explain what those two meanings are?
I don’t mean anything deep by it, just that for example a system might be able to optimize our environment to .99 human-optimal (which is pretty well approximated by the phrase “solving all problems”) and thereby create, say, a pervasive and crippling sense of ennui that it can’t yet resolve (which would constitute a “problem”). There’s no contradiction in that scenario; the illusion of contradiction is created entirely by the sloppiness of language.
I don’t think I follow; if the environment is .99 human-optimal, then that remaining .01 gap implies that there are some problems remain to be solved, however few or minor, right?
It might simply be impossible to solve all problems, because of conflicting dependencies.
Yes, I agree that the remaining .01 gap represents problems that remain to be solved, which implies that “solving all problems” doesn’t literally apply to that scenario. If you’re suggesting that therefore such a scenario isn’t well-enough approximated by the phrase “solving all problems” to justify the phrase’s use, we have different understandings of the level of justification required.
The problem is not that all problems might be solved at some point but that as long as we don’t turn ourselves into something similarly capable as the CEV process, then there exists an oracle that we could ask if we wanted to. The existence of such an oracle is that which is diminishing the value of research and discovery.
I agree that the existence of such an oracle closes off certain avenues of research and discovery.
I’m not sure that keeping those avenues of research and discovery open is particularly important, but I agree that it might be.
If it turns out that the availability such an oracle closing off certain avenues is an important human problem, it seems to follow that any system capable of and motivated to solve human problems will ensure that no such oracle is available.
You seem to be saying that research and discovery has some intrinsic value, in addition to the benefits of actually discovering things and understanding them. If so, what is this value ?
The only answer I can think of is something like, “learning about new avenues of research that the oracle had not yet explored”, but I’m not sure whether that makes sense or not—since the perfect oracle would explore every avenue of research, and an imperfect oracle would strive toward perfection (as long as the oracle is rational).
Well, the process of research and discovery can itself be enjoyable. That said, I don’t feel that there is a need to hold onto our current enjoyable activities if a CEV can create novel superior ways for us to have fun.
I would posit that divergent behaviors and approaches to the norms will still occur, despite the existence of such an oracle just for the sake of imagination, exploration, and the enjoyment of the process itself. Such oracle would also be aware of the existence of unknown future factors, and the benefits of diverse approaches to problems in the face of factors with unknown long term benefits and viability until certain processes has been executed. As you said, such an oracle would then try to explore every avenue of research, while still focusing on the ones deemed most likely to be fruitful. Such oracle should also be good on self-reflection, and able to question its own approaches and the various perspectives it is able to subsume. After all, isn’t self introspection and self reflections part of how one improve themselves?
Then there’s the Fun theory sequence that DSimon have posted about.