I’m not sure that regretting correct choices is a terrible downside, depending on how you think of regret and its effects.
If regret is just “feeling bad”, then you should just not feel bad for no reason. So don’t regret anything. Yeah.
If regret is “feeling bad as negative reinforcement”, then regretting things that are mistakes in hindsight (as opposed to correct choices that turned out bad) teaches you not to make such mistakes. Regretting all choices that led to bad outcomes hopefully will also teach this, if you correctly identify mistakes in hindsight, but this is a noisier (and slower) strategy.
If regret is “feeling bad, which makes you reconsider your strategy”, then you should regret everything that leads to a bad outcome, whether or not you think you made a mistake, because that is the only kind of strategy that can lead you to identify new kinds of mistakes you might be making.
If we don’t actually have a common understanding of what “regret” refers to, it’s probably best to stop using the term altogether.
If I’m always less likely to implement a given decision procedure D because implementing D in the past had a bad outcome, and always more likely to implement D because doing so had a good outcome (which is what I understand Quill_McGee to be endorsing, above), I run the risk of being less likely to implement a correct procedure as the result of a chance event.
There are more optimal approaches.
I endorse re-evaluating strategies in light of surprising outcomes.(It’s not necessarily a bad thing to do in the absence of surprising outcomes, but there’s usually something better to do with our time.) A bad outcome isn’t necessarily surprising—if I call “heads” and the coin lands tails, that’s bad, but unsurprising. If it happens twice, that’s bad and a little surprising. If it happens ten times, that’s bad and very surprising.
I was thinking of the “feeling bad and reconsider” meaning. That is, you don’t want regret to occur, so if you are systematically regretting your actions it might be time to try something new. Now, perhaps you were acting optimally already and when you changed you got even /more/ regret, but in that case you just switch back.
That’s true, but I think I agree with TheOtherDave that the things that should make you start reconsidering your strategy are not bad outcomes but surprising outcomes.
In many cases, of course, bad outcomes should be surprising. But not always: sometimes you choose options you expect to lose, because the payoff is sufficiently high. Plus, of course, you should reconsider your strategy when it succeeds for reasons you did not expect: if I make a bad move in chess, and my opponent does not notice, I still need to work on not making such a move again.
I also worry that relying on regret to change your strategy is vulnerable to loss aversion and similar bugs in human reasoning. Betting and losing $100 feels much more bad than betting and winning $100 feels good, to the extent that we can compare them. If you let your regret of the outcome decide your strategy, then you end up teaching yourself to use this buggy feeling when you make decisions.
Right. And your point about reconsidering strategy on surprising good outcomes is an important one. (My go-to example of this is usually the stranger who keeps losing bets on games of skill, but is surprisingly willing to keep betting larger and larger sums on the game anyway.)
I’m not sure that regretting correct choices is a terrible downside, depending on how you think of regret and its effects.
If regret is just “feeling bad”, then you should just not feel bad for no reason. So don’t regret anything. Yeah.
If regret is “feeling bad as negative reinforcement”, then regretting things that are mistakes in hindsight (as opposed to correct choices that turned out bad) teaches you not to make such mistakes. Regretting all choices that led to bad outcomes hopefully will also teach this, if you correctly identify mistakes in hindsight, but this is a noisier (and slower) strategy.
If regret is “feeling bad, which makes you reconsider your strategy”, then you should regret everything that leads to a bad outcome, whether or not you think you made a mistake, because that is the only kind of strategy that can lead you to identify new kinds of mistakes you might be making.
If we don’t actually have a common understanding of what “regret” refers to, it’s probably best to stop using the term altogether.
If I’m always less likely to implement a given decision procedure D because implementing D in the past had a bad outcome, and always more likely to implement D because doing so had a good outcome (which is what I understand Quill_McGee to be endorsing, above), I run the risk of being less likely to implement a correct procedure as the result of a chance event.
There are more optimal approaches.
I endorse re-evaluating strategies in light of surprising outcomes.(It’s not necessarily a bad thing to do in the absence of surprising outcomes, but there’s usually something better to do with our time.) A bad outcome isn’t necessarily surprising—if I call “heads” and the coin lands tails, that’s bad, but unsurprising. If it happens twice, that’s bad and a little surprising. If it happens ten times, that’s bad and very surprising.
I was thinking of the “feeling bad and reconsider” meaning. That is, you don’t want regret to occur, so if you are systematically regretting your actions it might be time to try something new. Now, perhaps you were acting optimally already and when you changed you got even /more/ regret, but in that case you just switch back.
That’s true, but I think I agree with TheOtherDave that the things that should make you start reconsidering your strategy are not bad outcomes but surprising outcomes.
In many cases, of course, bad outcomes should be surprising. But not always: sometimes you choose options you expect to lose, because the payoff is sufficiently high. Plus, of course, you should reconsider your strategy when it succeeds for reasons you did not expect: if I make a bad move in chess, and my opponent does not notice, I still need to work on not making such a move again.
I also worry that relying on regret to change your strategy is vulnerable to loss aversion and similar bugs in human reasoning. Betting and losing $100 feels much more bad than betting and winning $100 feels good, to the extent that we can compare them. If you let your regret of the outcome decide your strategy, then you end up teaching yourself to use this buggy feeling when you make decisions.
Right. And your point about reconsidering strategy on surprising good outcomes is an important one. (My go-to example of this is usually the stranger who keeps losing bets on games of skill, but is surprisingly willing to keep betting larger and larger sums on the game anyway.)