SI is a very narrowly focused institute. If you don’t buy the whole argument, there’s very little reason to donate. I’m not sure SI should dissolve, I think they can reform. It’s pretty obvious from their output that SI is essentially a machine ethics think tank. The obvious path to reform is greater pluralism and greater relevance to current debate. SI could focus on being the premiere machine ethics think tank, get involved in current ethical debates around the uses of AI, develop a more flexible ethical framework, and keep the Friendliness and Intelligence Explosion stuff as one possibility among many. This might allow them to grow and gain more resources (i.e., from the government, from military robotics companies wanting to appear responsible, etc), which would be a positive outcome for everyone. It’d also make it easier to donate, since instead of having to believe a narrow set of rather difficult to evaluate propositions, you’d simply have to value encouraging ethical debate around AI.
I generally expect that broad-focus organizations with a lot of resources and multiple constituencies will end up spending a LOT of their resources on internal status struggles. Given what little I’ve seen about SI’s skill and expertise at managing the internal politics of such an arrangement, I would expect the current staff to be promptly displaced by more skillful politicians if they went down this road, and the projects of interest to that staff to end up with even fewer resources than they have now.
Given what little I’ve seen about SI’s skill and expertise at managing the internal politics of such an arrangement, I would expect the current staff to be promptly displaced by more skillful politicians if they went down this road, and the projects of interest to that staff to end up with even fewer resources than they have now.
I think this has already happened to some extent. Reflective people who have good epistemic habits but who don’t get shit done have had their influence over SingInst policy taken away while lots of influence has been granted to people like Luke and Louie who get lots of shit done and who make the organization look a lot prettier but whose epistemic habits are, in my eyes, relatively suspect.
You’re probably correct. The current staff would have the same problem of establishing legitimacy they have now but within the context of the larger organisation.
SI is a very narrowly focused institute. If you don’t buy the whole argument, there’s very little reason to donate. I’m not sure SI should dissolve, I think they can reform. It’s pretty obvious from their output that SI is essentially a machine ethics think tank. The obvious path to reform is greater pluralism and greater relevance to current debate. SI could focus on being the premiere machine ethics think tank, get involved in current ethical debates around the uses of AI, develop a more flexible ethical framework, and keep the Friendliness and Intelligence Explosion stuff as one possibility among many. This might allow them to grow and gain more resources (i.e., from the government, from military robotics companies wanting to appear responsible, etc), which would be a positive outcome for everyone. It’d also make it easier to donate, since instead of having to believe a narrow set of rather difficult to evaluate propositions, you’d simply have to value encouraging ethical debate around AI.
I generally expect that broad-focus organizations with a lot of resources and multiple constituencies will end up spending a LOT of their resources on internal status struggles. Given what little I’ve seen about SI’s skill and expertise at managing the internal politics of such an arrangement, I would expect the current staff to be promptly displaced by more skillful politicians if they went down this road, and the projects of interest to that staff to end up with even fewer resources than they have now.
I think this has already happened to some extent. Reflective people who have good epistemic habits but who don’t get shit done have had their influence over SingInst policy taken away while lots of influence has been granted to people like Luke and Louie who get lots of shit done and who make the organization look a lot prettier but whose epistemic habits are, in my eyes, relatively suspect.
Could you expand a bit more on why you think the epistemic habits are suspect compared to previous staffers?
I think there’s an important lesson here about the relative importance of being able to get shit done versus good epistemic habits.
Are you talking about guys like e.g. Steve Rayhawk or Peter de Blanc?
You’re probably correct. The current staff would have the same problem of establishing legitimacy they have now but within the context of the larger organisation.
I disagree, and so apparently do some of SI’s major donors.