You and Swimmer963 are making the mistake of applying heroic responsibility only to optimising some local properties. Of course that will mean damaging the greater environment: applying “heroic responsibility” basically means you do your best AGI impression, so if you only optimise for a certain subset of your morality your results aren’t going to be pleasant.
Heroic responsibility only works if you take responsibility for everything. Not just the one patient you’re officially being held accountable for, not just the most likely Everett branches, not just the events you see with your own eyes. If your calling a halt to the human machine you are a part of truly has an expected negative effect, then it is your heroic responsibility to shut up and watch others make horrible mistakes.
A culture of heroic responsibility demands appropriate humility; it demands making damn sure what you’re doing is correct before defying your assigned duties. And if human psychology is such that punishing specific people for specific events works, then it is everyone’s heroic responsibility to make sure that rule exists.
Applying this in practice would, for most people, boil down to effective altruism: acquiring and pooling resources to enable a smaller group to optimise the world directly (after acquiring enough evidence of the group’s reliability that you know they’ll do a better job at it than you), trying to influence policy through political activism, and/or assorted meta-goals, all the while searching for ways to improve the system and obeying the law. Insisting you help directly instead of funding others would be statistical murder in the framework of heroic responsibility.
No: the concept that our ethics is utilitarian is independent from the concept that it is the only acceptable way of making decisions (where “acceptable” is an emotional/moral term).
What is an acceptable way of making decisions (where “acceptable” is an emotional/moral term) looks like an ethical question, how can it be independent from your ethics?
In ethics, the question would be answered by “yes, this ethical system is the only acceptable way to make decisions” by definition. In practice, this fact is not sufficient to make more than 0.01% of the world anywhere near heroically responsible (~= considering ethics the only emotionally/morally/role-followingly acceptable way of making decisions), so apparently the question is not decided by ethics.
Instead, roles and emotions play a large part in determining what is acceptable. In western society, the role of someone who is responsible for everything and not in the corresponding position of power is “the hero”. Yudkowsky (and HPJEV) might have chosen to be heroically responsible because he knows it is the consistent/rational conclusion of human morality and he likes being consistent/rational very much, or because he likes being a hero, or more likely a combination of both. The decision is made due to the role he wants to lead, not due to the ethics itself.
There are various types of consequentalism. The lack of distinction between ethical necessity and supererogation, and the general focus about optimizing the world, are typical of utilitarianism, which is in fact often associated with effective altruism (although it is not strictly necessary for it).
I think it applies to any and all of them just as well, but I (very stupidly) didn’t realize until now that utilitarianism is (a type of) consequentialism.
You and Swimmer963 are making the mistake of applying heroic responsibility only to optimising some local properties. Of course that will mean damaging the greater environment: applying “heroic responsibility” basically means you do your best AGI impression, so if you only optimise for a certain subset of your morality your results aren’t going to be pleasant.
Heroic responsibility only works if you take responsibility for everything. Not just the one patient you’re officially being held accountable for, not just the most likely Everett branches, not just the events you see with your own eyes. If your calling a halt to the human machine you are a part of truly has an expected negative effect, then it is your heroic responsibility to shut up and watch others make horrible mistakes.
A culture of heroic responsibility demands appropriate humility; it demands making damn sure what you’re doing is correct before defying your assigned duties. And if human psychology is such that punishing specific people for specific events works, then it is everyone’s heroic responsibility to make sure that rule exists.
Applying this in practice would, for most people, boil down to effective altruism: acquiring and pooling resources to enable a smaller group to optimise the world directly (after acquiring enough evidence of the group’s reliability that you know they’ll do a better job at it than you), trying to influence policy through political activism, and/or assorted meta-goals, all the while searching for ways to improve the system and obeying the law. Insisting you help directly instead of funding others would be statistical murder in the framework of heroic responsibility.
So “heroic responsibility” just means “total utilitarianism”?
No: the concept that our ethics is utilitarian is independent from the concept that it is the only acceptable way of making decisions (where “acceptable” is an emotional/moral term).
What is an acceptable way of making decisions (where “acceptable” is an emotional/moral term) looks like an ethical question, how can it be independent from your ethics?
In ethics, the question would be answered by “yes, this ethical system is the only acceptable way to make decisions” by definition. In practice, this fact is not sufficient to make more than 0.01% of the world anywhere near heroically responsible (~= considering ethics the only emotionally/morally/role-followingly acceptable way of making decisions), so apparently the question is not decided by ethics.
Instead, roles and emotions play a large part in determining what is acceptable. In western society, the role of someone who is responsible for everything and not in the corresponding position of power is “the hero”. Yudkowsky (and HPJEV) might have chosen to be heroically responsible because he knows it is the consistent/rational conclusion of human morality and he likes being consistent/rational very much, or because he likes being a hero, or more likely a combination of both. The decision is made due to the role he wants to lead, not due to the ethics itself.
It just means ‘consequentalism’.
There are various types of consequentalism. The lack of distinction between ethical necessity and supererogation, and the general focus about optimizing the world, are typical of utilitarianism, which is in fact often associated with effective altruism (although it is not strictly necessary for it).
I think it applies to any and all of them just as well, but I (very stupidly) didn’t realize until now that utilitarianism is (a type of) consequentialism.