With regard to discussion, I just implemented what I was instructed to do.
“I was only following orders” works less well for engineers than it did for Nazis. As Don Gotterbarn or Richard Volkman would have us remember, programming exactly to specs is not doing your job well.
Downvoted for unkindness—no one likes to be involved in a comparison where Nazis are the baseline.
“Programming exactly to specs” is part of the job, but seeing to it that the specs have desirable outcomes when implemented also is, so I partly agree—I think you could do a better job of evangelizing here.
You might want to check out the User Manifesto, a work in progress.
wmoore doesn’t appear to be a normal LW member; the only comments I see in xyr history are about code changes, so I suspect xe’s simply the person who implements the changes that Eliezer (or whoever has the relevant authority) requests. It doesn’t make sense to me, if that’s true, to make wmoore responsible for confirming that those changes have been run by the group.
The engineers of the software for the Therac-25 were simply implementing a spec. It caused people to die because they did not look at the bigger picture. Something all software engineers should be aware of.
Just doing my “The more you know” moment as a professional computer ethicist.
Is the programmer always obligated to personally confirm that all potentially-affected parties have had an opportunity to comment on the intended result? That seems like a much higher standard than just checking to make sure the spec will do what the customer intends it to do.
Is the programmer always obligated to personally confirm that all potentially-affected parties have had an opportunity to comment on the intended result
No, but the programmer is obligated to personally confirm that all relevant stakeholders have been considered in the analysis of the ethical impacts of their work.
In this particular case, it seems obvious that major changes require discussion, or at least that “I just implemented what I was instructed to do” is not a proper response.
We’re disagreeing mostly on the “personally confirm” point, I believe—I don’t believe that that’s true in all cases.
I don’t think it’s unreasonable for wmoore to have assumed that Eliezer had taken care of the moral analysis of this case. (I wouldn’t disagree with you if you were suggesting that Eliezer should have checked with us.) Wmoore doesn’t obviously know how this group is structured, or what kinds of things we would expect to be consulted about; most online groups are structured in a way that relies much more on the low cost of leaving the group and finding another one to act as a balance for the group owner’s considerable power, which makes the assumption that the owner’s power should be supported a reasonable one in most cases. (Do you also object to making add-on modules for PhP forums available for forum owners to use with no oversight?)
I would agree that, in general, programmers (and anyone involved in a project) should make sure that there’s a balance of power, or that someone is considering the moral implications of the project, or both. In this class of cases (management decisions affecting free, open online groups), I believe that the presence of the former is sufficient. In many of the situations I program for, it’s not, and I do make a point of considering the implications of any spec I implement—both because it’s part of my job description, and because it’s a correct thing to do.
We’re disagreeing mostly on the “personally confirm” point, I believe—I don’t believe that that’s true in all cases.
Are we still in disagreement?
Indeed, I think that’s the sticking point. Perhaps it would be okay to sometimes offload the moral analysis of a project to a professional who explicitly is doing that (like your company’s ethicist, or a manager who has certified that he’s done some standard analysis) but it’s never okay to simply assume it’s been done.
You give advice that doesn’t apply in the particular case, and rarely in general.
I’m not sure what you mean by this. Do you think that in general, it’s not appropriate for programmers to consider the wider impacts of the code they write?
Do you think that in general, it’s not appropriate for programmers to consider the wider impacts of the code they write?
It is rarely important from the moral standpoint, since most things that programmers write (or things that most programmers write) don’t have knowable moral impact. On the other hand, being aware of the wider context may improve quality of the result.
On the other hand, being aware of the wider context may improve quality of the result.
Isn’t “improving quality” a value for a good computer professional?
It is rarely important from the moral standpoint, since most things that programmers write (or things that most programmers write) don’t have knowable moral impact.
I disagree, but I don’t have data on the sorts of things most programmers write. In my experience, computer programs impact the lives of people (or why would you write them?) and therefore almost universally should be considered from the moral standpoint.
I do associate primarily with programmers in the ‘hot topic’ fields for computer ethics—web surveys, data mining, military applications—so that may skew my perceptions. It’s significantly easier to see the connection between ethics and one’s work when one is working on the system a robotic soldier uses to determine how much damage it’s allowed to cause to a nearby hospital.
But I would argue that this blog is either in that category or not performing its function. Eliezer argued that it was worth his time to work on this place because it contributed to his (and the SIAI’s) mission that will change the shape of the future of humanity. If that’s true, even minor changes here can be expected to have more of a ‘moral impact’ than most things, military robots included.
Yes an up vote on a post is worth 10 karma points to the contributor, a down vote −10 points.
With regard to discussion, I just implemented what I was instructed to do.
“I was only following orders” works less well for engineers than it did for Nazis. As Don Gotterbarn or Richard Volkman would have us remember, programming exactly to specs is not doing your job well.
Downvoted for unkindness—no one likes to be involved in a comparison where Nazis are the baseline.
“Programming exactly to specs” is part of the job, but seeing to it that the specs have desirable outcomes when implemented also is, so I partly agree—I think you could do a better job of evangelizing here.
You might want to check out the User Manifesto, a work in progress.
wmoore doesn’t appear to be a normal LW member; the only comments I see in xyr history are about code changes, so I suspect xe’s simply the person who implements the changes that Eliezer (or whoever has the relevant authority) requests. It doesn’t make sense to me, if that’s true, to make wmoore responsible for confirming that those changes have been run by the group.
The engineers of the software for the Therac-25 were simply implementing a spec. It caused people to die because they did not look at the bigger picture. Something all software engineers should be aware of.
Just doing my “The more you know” moment as a professional computer ethicist.
Is the programmer always obligated to personally confirm that all potentially-affected parties have had an opportunity to comment on the intended result? That seems like a much higher standard than just checking to make sure the spec will do what the customer intends it to do.
No, but the programmer is obligated to personally confirm that all relevant stakeholders have been considered in the analysis of the ethical impacts of their work.
In this particular case, it seems obvious that major changes require discussion, or at least that “I just implemented what I was instructed to do” is not a proper response.
We’re disagreeing mostly on the “personally confirm” point, I believe—I don’t believe that that’s true in all cases.
I don’t think it’s unreasonable for wmoore to have assumed that Eliezer had taken care of the moral analysis of this case. (I wouldn’t disagree with you if you were suggesting that Eliezer should have checked with us.) Wmoore doesn’t obviously know how this group is structured, or what kinds of things we would expect to be consulted about; most online groups are structured in a way that relies much more on the low cost of leaving the group and finding another one to act as a balance for the group owner’s considerable power, which makes the assumption that the owner’s power should be supported a reasonable one in most cases. (Do you also object to making add-on modules for PhP forums available for forum owners to use with no oversight?)
I would agree that, in general, programmers (and anyone involved in a project) should make sure that there’s a balance of power, or that someone is considering the moral implications of the project, or both. In this class of cases (management decisions affecting free, open online groups), I believe that the presence of the former is sufficient. In many of the situations I program for, it’s not, and I do make a point of considering the implications of any spec I implement—both because it’s part of my job description, and because it’s a correct thing to do.
Are we still in disagreement?
Indeed, I think that’s the sticking point. Perhaps it would be okay to sometimes offload the moral analysis of a project to a professional who explicitly is doing that (like your company’s ethicist, or a manager who has certified that he’s done some standard analysis) but it’s never okay to simply assume it’s been done.
You give advice that doesn’t apply in the particular case, and rarely in general.
I’m not sure what you mean by this. Do you think that in general, it’s not appropriate for programmers to consider the wider impacts of the code they write?
It is rarely important from the moral standpoint, since most things that programmers write (or things that most programmers write) don’t have knowable moral impact. On the other hand, being aware of the wider context may improve quality of the result.
Isn’t “improving quality” a value for a good computer professional?
I disagree, but I don’t have data on the sorts of things most programmers write. In my experience, computer programs impact the lives of people (or why would you write them?) and therefore almost universally should be considered from the moral standpoint.
I do associate primarily with programmers in the ‘hot topic’ fields for computer ethics—web surveys, data mining, military applications—so that may skew my perceptions. It’s significantly easier to see the connection between ethics and one’s work when one is working on the system a robotic soldier uses to determine how much damage it’s allowed to cause to a nearby hospital.
But I would argue that this blog is either in that category or not performing its function. Eliezer argued that it was worth his time to work on this place because it contributed to his (and the SIAI’s) mission that will change the shape of the future of humanity. If that’s true, even minor changes here can be expected to have more of a ‘moral impact’ than most things, military robots included.