While some people tried to appeal to non-linear aggregation, you would have to appeal to a non-linear aggregation which was non-linear enough to reduce 3^^^3 to a small constant.
So an algorithm like, “order utilities from least to greatest, then sum with a weight if 1/n^2, where n is their position in the list” could pick dust specks over torture while recommending most people not go sky diving (as their benefit is outweighed by the detriment to those less fortunate).
This would mean that scope insensitivity, beyond a certain point, is a feature of our morality rather than a bias; I am not sure my opinion of this outcome.
That said, while giving an answer to the one problem that some seem more comfortable with, and to the second that everyone agrees on, I expect there are clear failure modes I haven’t thought of.
Edited to add:
This of course holds for weights of 1/n^a for any a>1; the most convincing defeat of this proposition would be showing that weights of 1/n (or 1/(n log(n))) drop off quickly enough to lead to bad behavior.
On recently encountering the wikipedia page on Utility Monsters and thence to the Mere Addition Paradox, it occurs to me that this seems to neatly defang both.
Edited—rather, completely defangs the Mere Addition Paradox, may or may not completely defang Utility Monsters depending on details but at least reduces their impact.
Sum(1/n^2, 1, 3^^^3) < Sum(1/n^2, 1, inf) = (pi^2)/6
So an algorithm like, “order utilities from least to greatest, then sum with a weight if 1/n^2, where n is their position in the list” could pick dust specks over torture while recommending most people not go sky diving (as their benefit is outweighed by the detriment to those less fortunate).
This would mean that scope insensitivity, beyond a certain point, is a feature of our morality rather than a bias; I am not sure my opinion of this outcome.
That said, while giving an answer to the one problem that some seem more comfortable with, and to the second that everyone agrees on, I expect there are clear failure modes I haven’t thought of.
Edited to add:
This of course holds for weights of 1/n^a for any a>1; the most convincing defeat of this proposition would be showing that weights of 1/n (or 1/(n log(n))) drop off quickly enough to lead to bad behavior.
On recently encountering the wikipedia page on Utility Monsters and thence to the Mere Addition Paradox, it occurs to me that this seems to neatly defang both.
Edited—rather, completely defangs the Mere Addition Paradox, may or may not completely defang Utility Monsters depending on details but at least reduces their impact.