It might turn out that what does satisfy us is to be “free”, to do what we want, even if that means that we will mess up our own future. It might turn out that humans are only satisfied if they can work on existential problems, “no risk no fun”. Or we might simply want to learn about the nature of reality. The mere existence of an FAI might spoil all of it. Would you care to do science if there was some AI-God that already knew all the answers? Would you be satisfied if it didn’t tell you the answers or made you forget that it does exist so that you’d try to invent AGI without ever succeeding?
But there is another possible end. Even today many people are really bored and don’t particularly enjoy life. What if it turns out that there is no “right” out there or that it can be reached fairly easily without any way to maximize it further. In other words, what if fun is something that isn’t infinite but a goal that can be reached? What if it all turns out to be wireheading, the only difference between 10 minutes of wireheading or 10^1000 years being the number enumerating the elapsed time? Think about it, would you care about 10^1000 years of inaction? What would you do if that was the optimum? Maybe we’ll just decide to choose the void instead.
This is a different context for satisfaction, but to answer your questions:
yes, I often find satisfying working through problems that have already been solved, though I appreciate that not everyone does;
no, I would not want to be denied the solutions if I asked (assuming there isn’t some other reason why giving me the solution is harmful), or kept in ignorance of the existence of those solutions (ibid);
if it turns out that all of my desires as they currently exist are fully implemented, leaving me with no room for progress and no future prospects better than endless joy, fulfillment and satisfaction, I’d be satisfied and fulfilled and joyful.
Admittedly, I might eventually become unsatisfied with that and desire something else, at which point I would devote efforts to satisfying that new desire. It doesn’t seem terribly likely that my non-existence would be the best possible way of doing so, but I suppose it’s possible, and if it happened I would cease to exist.
It might turn out that what does satisfy us is to be “free”, to do what we want, even if that means that we will mess up our own future. It might turn out that humans are only satisfied if they can work on existential problems, “no risk no fun”. Or we might simply want to learn about the nature of reality. The mere existence of an FAI might spoil all of it. Would you care to do science if there was some AI-God that already knew all the answers? Would you be satisfied if it didn’t tell you the answers or made you forget that it does exist so that you’d try to invent AGI without ever succeeding?
But there is another possible end. Even today many people are really bored and don’t particularly enjoy life. What if it turns out that there is no “right” out there or that it can be reached fairly easily without any way to maximize it further. In other words, what if fun is something that isn’t infinite but a goal that can be reached? What if it all turns out to be wireheading, the only difference between 10 minutes of wireheading or 10^1000 years being the number enumerating the elapsed time? Think about it, would you care about 10^1000 years of inaction? What would you do if that was the optimum? Maybe we’ll just decide to choose the void instead.
This is a different context for satisfaction, but to answer your questions:
yes, I often find satisfying working through problems that have already been solved, though I appreciate that not everyone does;
no, I would not want to be denied the solutions if I asked (assuming there isn’t some other reason why giving me the solution is harmful), or kept in ignorance of the existence of those solutions (ibid);
if it turns out that all of my desires as they currently exist are fully implemented, leaving me with no room for progress and no future prospects better than endless joy, fulfillment and satisfaction, I’d be satisfied and fulfilled and joyful.
Admittedly, I might eventually become unsatisfied with that and desire something else, at which point I would devote efforts to satisfying that new desire. It doesn’t seem terribly likely that my non-existence would be the best possible way of doing so, but I suppose it’s possible, and if it happened I would cease to exist.
It might indeed.