I think this comment is getting enough vote & discussion heat for me to feel the merit in clarifying with the following statements:
Most of the value in the world is created by ambitious people. It’s not evil to have strong preferences.
If the average ML engineer or entrepreneur had the ethics-oracle of Connor Leahy, we would be in much better shape. Most seem to be either not f(curious,honest) enough or principled enough to even make a plausible attempt at not ending the world. Sam Altman needs to hear the above lecture-rant 10,000 times before Connor needs to hear it once. The reason I write this comment on a post about Connor Leahy is solely because of convenience, because I can’t actually talk to Sam Altman, and because I felt like writing one after reading the above post and not because he’s even within 3 orders of magnitude of the worst examples of this behavior.
I strong downvoted your comment in both dimensions because I found it disagreeable and counterproductive. This kind of “Kremlinology of the heart” is toxic and demoralizing. It’s why I never ever bother to do anything motivated by altruism: because I know when I start trying to do the right thing, I’ll get attacked by people who think they know what’s in my heart. When I openly act in selfish self-interest, nobody has anything to say about it, but any time I do selfless things, people start questioning my motives; it’s clear what I’m incentivized to do. If you really want people to do good thing, don’t play status games like this. Incentivize the behavior you want.
I feel unsure about the merits of this for other contexts (because it can indeed create a toxic atmosphere), but I think there are specific contexts where scrutinizing someone’s decision-making algorithm seems particularly important:
Somewhat unilateral pursuit of an activity with high downside risk
Position of high influence without much accountability or legibility for outsiders to give useful criticism
Heading an alignment organization with strong information security where you have enough control so that it’s unusual compared to other organizations fulfils both criteria.
So, I’d say that not discussing the topic in contexts similar to this one would be a mistake.
I strong downvoted your comment in both dimensions because I found it disagreeable and counterproductive.
Generally, I think it would be net-negative to discourage such open discussions about unilateral, high-risk interventions—within the EA/AIS communities—that involve conflicts of interest. Especially, for example, unilateral interventions to create/fund for-profit AGI companies, or to develop/disseminate AI capabilities.
You know what, I’ve retracted the comment because frankly you’re probably right. Even if what I said is literally true, attacking the decision making architecture of Connor Leahy when he’s basically doing something good is not two of (true, kind, necessary). It makes people sad, it’s the kind of asymmetric justice thing I hate, and I don’t want to normalize it. Even when it’s attached with disclaimers or say “I’m just attacking you just cuz bro don’t take it personal.”
Most VC-types are easier to get a hold of than you think. They’re sort of in the business of being easy to get a hold of by smart weirdos. If you think you have something to say to him that might change his mind, there’s a good shot he’ll read your cold email.
I think this comment is getting enough vote & discussion heat for me to feel the merit in clarifying with the following statements:
Most of the value in the world is created by ambitious people. It’s not evil to have strong preferences.
If the average ML engineer or entrepreneur had the ethics-oracle of Connor Leahy, we would be in much better shape. Most seem to be either not f(curious,honest) enough or principled enough to even make a plausible attempt at not ending the world. Sam Altman needs to hear the above lecture-rant 10,000 times before Connor needs to hear it once. The reason I write this comment on a post about Connor Leahy is solely because of convenience, because I can’t actually talk to Sam Altman, and because I felt like writing one after reading the above post and not because he’s even within 3 orders of magnitude of the worst examples of this behavior.
I strong downvoted your comment in both dimensions because I found it disagreeable and counterproductive. This kind of “Kremlinology of the heart” is toxic and demoralizing. It’s why I never ever bother to do anything motivated by altruism: because I know when I start trying to do the right thing, I’ll get attacked by people who think they know what’s in my heart. When I openly act in selfish self-interest, nobody has anything to say about it, but any time I do selfless things, people start questioning my motives; it’s clear what I’m incentivized to do. If you really want people to do good thing, don’t play status games like this. Incentivize the behavior you want.
I feel unsure about the merits of this for other contexts (because it can indeed create a toxic atmosphere), but I think there are specific contexts where scrutinizing someone’s decision-making algorithm seems particularly important:
Somewhat unilateral pursuit of an activity with high downside risk
Position of high influence without much accountability or legibility for outsiders to give useful criticism
Heading an alignment organization with strong information security where you have enough control so that it’s unusual compared to other organizations fulfils both criteria.
So, I’d say that not discussing the topic in contexts similar to this one would be a mistake.
I’d add to that bullet list:
Severe conflicts of interest are involved.
Generally, I think it would be net-negative to discourage such open discussions about unilateral, high-risk interventions—within the EA/AIS communities—that involve conflicts of interest. Especially, for example, unilateral interventions to create/fund for-profit AGI companies, or to develop/disseminate AI capabilities.
You know what, I’ve retracted the comment because frankly you’re probably right. Even if what I said is literally true, attacking the decision making architecture of Connor Leahy when he’s basically doing something good is not two of (true, kind, necessary). It makes people sad, it’s the kind of asymmetric justice thing I hate, and I don’t want to normalize it. Even when it’s attached with disclaimers or say “I’m just attacking you just cuz bro don’t take it personal.”
Thanks!
Most VC-types are easier to get a hold of than you think. They’re sort of in the business of being easy to get a hold of by smart weirdos. If you think you have something to say to him that might change his mind, there’s a good shot he’ll read your cold email.