You and Swimmer963 are making the mistake of applying heroic responsibility only to optimising some local properties. Of course that will mean damaging the greater environment: “heroic responsibility” basically means you do your best AGI impression, so if you only optimise for a certain subset of your morality your results aren’t going to be pleasant.
Heroic responsibility only works if you take responsibility for everything. Not just the one patient you’re officially being held accountable for, not just the most likely Everett branches, not just the events you see with your own eyes. If your calling a halt to the human machine you are a part of truly has an expected negative effect, then it is your heroic responsibility to shut up and watch others make horrible mistakes.
A culture of heroic responsibility demands appropriate humility; it demands making damn sure what you’re doing is correct before defying your assigned duties. And if human psychology is such that a criminal justice system is still appropriate (where specific individuals are punished for specific events), then it is everyone’s job to make sure there is a criminal justice system.
You and Swimmer963 are making the mistake of applying heroic responsibility only to optimising some local properties. Of course that will mean damaging the greater environment: “heroic responsibility” basically means you do your best AGI impression, so if you only optimise for a certain subset of your morality your results aren’t going to be pleasant.
Heroic responsibility only works if you take responsibility for everything. Not just the one patient you’re officially being held accountable for, not just the most likely Everett branches, not just the events you see with your own eyes. If your calling a halt to the human machine you are a part of truly has an expected negative effect, then it is your heroic responsibility to shut up and watch others make horrible mistakes.
A culture of heroic responsibility demands appropriate humility; it demands making damn sure what you’re doing is correct before defying your assigned duties. And if human psychology is such that a criminal justice system is still appropriate (where specific individuals are punished for specific events), then it is everyone’s job to make sure there is a criminal justice system.
Applying this in practice