I think then we just fundamentally disagree with the ethical role of CEO in the company. I believe that it is to find and gather people who are engaged with the arguments of the critic’s (like that guy from this forum who was hired by Anthropic). If you have people on your side who are able to engage with the arguments, then this is good enough for me. I don’t see the role of CEO is publicly engaging with critic’s arguments even in the moral sense. In the moral sense, my requirements would actually be even lesser. IMO, it would be enough just to have people broadly on your side (optimists for example) to engage with the critics.
I believe the disagreement is not about CEOs, it’s about illegitimate power. If you’ll allow me a brief detour, I’ll try to explain.
Sometimes people grant other people power over them. For instance, I have agreed to work at my company. I’ve agreed that my CEO can fire me, and make many other demands of me, in exchange for money and other various demands I can make of him. Ideally we entered into this agreement freely and without inappropriate pressure.
Other times, people get power over people without any agreement or granting. Your parent typically has a lot of power over you until you are 18. They can determine what you eat, where you are physically located, what privacy you have, what resources you have, etc. Also, as has been very important for most of history, people have been able to be physically violent to one another and hurt people or even end their lives. Neither of these powers are come to consensually.
For the latter, an important question to ask is “How does one wield this power well? What does it mean to wield it well vs poorly?” There are many ways to parent, many choices about diet and schooling and sleep times and what are fair punishment. But some parents starve their children and beat them for not following instructions and sexually assault them. This is an inappropriate use of power.
There’s a legitimacy that comes by being granted power, and an illegitimacy that comes with getting or wielding power that you were not granted.
I think that there’s a big question about how to wield it well vs poorly, and how to respect people you have illegitimate powers over. Something I believe is that society functions better if we take seriously the attempt to wield it well. To not casually kill someone if you can get away with it and feel like it, but consider them as people worthy of respect, and ask how you can respect the people you’ve been non-consensually given power over.
This requires doing some work. It involves asking yourself what’s a reasonable amount of effort to spend modeling their preferences given how much power you have over someone, it involves asking yourself if society has any good received wisdom on what to do with this particular power, and it involves engaging with people who are aggrieved by your use of power over them.
Now, the standard model for companies and businesses is a libtertarian-esque free market, where all trades are consensual and have no inappropriate pressure. This is like the first situation I describe, where a company has no people it has undue power over, no people who it can treat better or worse with the power it has over them.
The situation where you are building machines you believe may kill literally everyone, is like the second situation, where you have a very different power dynamic, where you’re making choices that affect everyone’s lives and that they had little-to-no say in. In such a situation, I think if you are going to do what is good and right, you owe it to show up and engage with those who believe you are using the power you have over them in ways that are seriously hurting them.
That’s the difference between this CEO situation and all of the others. It’s not about standards for CEOs, its about standards for illegitimate power.
This kind of talking-with-the-aggrieved-people-you-have-immense-power-over is a way of showing the people basic respect, and it is not present in this case. I believe these people are risking my life and many others’, and they seem to me disrespectful and largely uninterested in showing up to talk with the people whose lives they are risking.
Then we are actually broadly in agreement. I just think that instead of CEOs responding to the public, having anyone at their side (the side of AI alignment being possible) responding is enough. Just as an example that I came up with, if a critic says that some detail is a reason for why AI will be dangerous, I do agree that someone needs to respond to the argument. But I would be fine with it being someone other than the CEO.
That’s why I am relatively optimistic about Anthropic hiring the guy who has been engaged with critic’s argument for years.
I think then we just fundamentally disagree with the ethical role of CEO in the company. I believe that it is to find and gather people who are engaged with the arguments of the critic’s (like that guy from this forum who was hired by Anthropic). If you have people on your side who are able to engage with the arguments, then this is good enough for me. I don’t see the role of CEO is publicly engaging with critic’s arguments even in the moral sense. In the moral sense, my requirements would actually be even lesser. IMO, it would be enough just to have people broadly on your side (optimists for example) to engage with the critics.
I believe the disagreement is not about CEOs, it’s about illegitimate power. If you’ll allow me a brief detour, I’ll try to explain.
Sometimes people grant other people power over them. For instance, I have agreed to work at my company. I’ve agreed that my CEO can fire me, and make many other demands of me, in exchange for money and other various demands I can make of him. Ideally we entered into this agreement freely and without inappropriate pressure.
Other times, people get power over people without any agreement or granting. Your parent typically has a lot of power over you until you are 18. They can determine what you eat, where you are physically located, what privacy you have, what resources you have, etc. Also, as has been very important for most of history, people have been able to be physically violent to one another and hurt people or even end their lives. Neither of these powers are come to consensually.
For the latter, an important question to ask is “How does one wield this power well? What does it mean to wield it well vs poorly?” There are many ways to parent, many choices about diet and schooling and sleep times and what are fair punishment. But some parents starve their children and beat them for not following instructions and sexually assault them. This is an inappropriate use of power.
There’s a legitimacy that comes by being granted power, and an illegitimacy that comes with getting or wielding power that you were not granted.
I think that there’s a big question about how to wield it well vs poorly, and how to respect people you have illegitimate powers over. Something I believe is that society functions better if we take seriously the attempt to wield it well. To not casually kill someone if you can get away with it and feel like it, but consider them as people worthy of respect, and ask how you can respect the people you’ve been non-consensually given power over.
This requires doing some work. It involves asking yourself what’s a reasonable amount of effort to spend modeling their preferences given how much power you have over someone, it involves asking yourself if society has any good received wisdom on what to do with this particular power, and it involves engaging with people who are aggrieved by your use of power over them.
Now, the standard model for companies and businesses is a libtertarian-esque free market, where all trades are consensual and have no inappropriate pressure. This is like the first situation I describe, where a company has no people it has undue power over, no people who it can treat better or worse with the power it has over them.
The situation where you are building machines you believe may kill literally everyone, is like the second situation, where you have a very different power dynamic, where you’re making choices that affect everyone’s lives and that they had little-to-no say in. In such a situation, I think if you are going to do what is good and right, you owe it to show up and engage with those who believe you are using the power you have over them in ways that are seriously hurting them.
That’s the difference between this CEO situation and all of the others. It’s not about standards for CEOs, its about standards for illegitimate power.
This kind of talking-with-the-aggrieved-people-you-have-immense-power-over is a way of showing the people basic respect, and it is not present in this case. I believe these people are risking my life and many others’, and they seem to me disrespectful and largely uninterested in showing up to talk with the people whose lives they are risking.
Then we are actually broadly in agreement. I just think that instead of CEOs responding to the public, having anyone at their side (the side of AI alignment being possible) responding is enough. Just as an example that I came up with, if a critic says that some detail is a reason for why AI will be dangerous, I do agree that someone needs to respond to the argument. But I would be fine with it being someone other than the CEO.
That’s why I am relatively optimistic about Anthropic hiring the guy who has been engaged with critic’s argument for years.