Our current methods might turn out to be biased in new and unexpected ways. Pascal’s mugging, the Lifespan Dilemma, blackmailing and the wrath of Löb’s theorem are just a few examples on how an agent build according to our current understanding of rationality could fail.
What are you trying to do here? Are you trying to give specific examples of cases in which doing the rational thing could be the wrong thing to do? Surely not, that would be oxymoronic—if you already know that the ‘rational thing’ is a mistake then it isn’t the rational thing. Failing that, how is mentioning these problems supposed to support “discounting extraordinary implications”? Rather than being cases where the logical implications are extraordinary they seem to just be subjects that some readers may find difficult or confusing to think about and which you want readers to associate with the opposing view.
Are you trying to give specific examples of cases in which doing the rational thing could be the wrong thing to do? Surely not, that would be oxymoronic—if you already know that the ‘rational thing’ is a mistake then it isn’t the rational thing.
Rational acts can be wrong—if the agent doesn’t already know that. This happens regularly in Bayesian hell.
Rational acts can be wrong—if the agent doesn’t already know that.
The point here is that giving examples of where in the future you already know that doing the rational thing to be wrong—that is what is absurd. If you already know what not to do then you don’t do it.
The OP’s point was that the “correct” actions were wrong accoriding to our current understanding of rationality—and his conclusion was that our current understanding of rationality might be wrong.
I wrote that our current understanding of rationality is not the last word and that we should therefore take account of model uncertainty.
If that was the extent of what you wrote I would not have commented. In this case I replied to this:
Pascal’s mugging, the Lifespan Dilemma, blackmailing and the wrath of Löb’s theorem are just a few examples on how an agent build according to our current understanding of rationality could fail.
Giving those as examples implies you are saying something more than “our current understanding of rationality is the last word”. Rejecting the position that argument supports is not nitpicking on definitions!
If our current understanding is that something is the wrong thing to do then our current understand of rationality doesn’t do it.
Oh boy...I just got what you are doing here. Nitpicking on a definition. Okay...of course rationality is winning and winning is doing what’s right according to your utility-function. What I meant is obviously that our methods are not perfect at guiding us and satisfying our utility-functions.
I am trying to hint at the possibility that our methods might be mathematically justified but that they might lead to unexpected side-effects when applied by computationally bounded agents under extreme circumstances, as some of our thought experiments indicate.
...if you already know that the ‘rational thing’ is a mistake then it isn’t the rational thing.
Our methods are the best we have and they work perfectly well on most problems we encounter. I am saying that we should discount some of the associated utility implications if we encounter edge cases. Ignoring the the implications would be irrational but taking them at face value wouldn’t be wise either.
What are you trying to do here? Are you trying to give specific examples of cases in which doing the rational thing could be the wrong thing to do? Surely not, that would be oxymoronic—if you already know that the ‘rational thing’ is a mistake then it isn’t the rational thing. Failing that, how is mentioning these problems supposed to support “discounting extraordinary implications”? Rather than being cases where the logical implications are extraordinary they seem to just be subjects that some readers may find difficult or confusing to think about and which you want readers to associate with the opposing view.
Rational acts can be wrong—if the agent doesn’t already know that. This happens regularly in Bayesian hell.
The point here is that giving examples of where in the future you already know that doing the rational thing to be wrong—that is what is absurd. If you already know what not to do then you don’t do it.
The OP’s point was that the “correct” actions were wrong accoriding to our current understanding of rationality—and his conclusion was that our current understanding of rationality might be wrong.
The OP is wrong. If our current understanding is that something is the wrong thing to do then our current understand of rationality doesn’t do it.
And that conclusion may be right, despite the argument being wrong.
I wrote that our current understanding of rationality is not the last word and that we should therefore take account of model uncertainty.
If that was the extent of what you wrote I would not have commented. In this case I replied to this:
Giving those as examples implies you are saying something more than “our current understanding of rationality is the last word”. Rejecting the position that argument supports is not nitpicking on definitions!
Oh boy...I just got what you are doing here. Nitpicking on a definition. Okay...of course rationality is winning and winning is doing what’s right according to your utility-function. What I meant is obviously that our methods are not perfect at guiding us and satisfying our utility-functions.
Not even remotely.
I am trying to hint at the possibility that our methods might be mathematically justified but that they might lead to unexpected side-effects when applied by computationally bounded agents under extreme circumstances, as some of our thought experiments indicate.
Our methods are the best we have and they work perfectly well on most problems we encounter. I am saying that we should discount some of the associated utility implications if we encounter edge cases. Ignoring the the implications would be irrational but taking them at face value wouldn’t be wise either.