If eventually it becomes clear that building an antinatalist AI is the right thing to do, I think the best way to accomplish it would be through an organization that’s like SIAI but isn’t too attached to the idea of FAI and just wants to do whatever is best.
Yes.
I, for one, am ultimately concerned with doing whatever’s best. I’m not wedded to doing FAI, and am certainly not wedded to doing 9-researchers-in-a-basement FAI.
I, for one, am ultimately concerned with doing whatever’s best. I’m not wedded to doing FAI, and am certainly not wedded to doing 9-researchers-in-a-basement FAI.
Well, that’s great. Still, there are quite a few problems.
How do I know
… that SI does not increase existential risk by solving problems that can be used to build AGI earlier?
… that you won’t launch a half-baked friendly AI that will turn the world into a hell?
… that you don’t implement some strategies that will do really bad things to some people, e.g. myself?
Every time I see a video of one of you people I think, “Wow, those seem like really nice people. I am probably wrong. They are going to do the right thing.”
But seriously, is that enough? Can I trust a few people with the power to shape the whole universe? Can I trust them enough to actually give them money? Can I trust them enough with my life until the end of the universe?
You can’t even tell me what “best” or “right” or “winning” stands for. How do I know that it can be or will be defined in a way that those labels will apply to me as well?
I have no idea what your plans are for the day when time runs out. I just hope that you are not going to hope for the best and run some not quite friendly AI that does really crappy things. I hope you consider the possibility of rather blowing everything up than risking even worse outcomes.
Yes.
I, for one, am ultimately concerned with doing whatever’s best. I’m not wedded to doing FAI, and am certainly not wedded to doing 9-researchers-in-a-basement FAI.
Well, that’s great. Still, there are quite a few problems.
How do I know
… that SI does not increase existential risk by solving problems that can be used to build AGI earlier?
… that you won’t launch a half-baked friendly AI that will turn the world into a hell?
… that you don’t implement some strategies that will do really bad things to some people, e.g. myself?
Every time I see a video of one of you people I think, “Wow, those seem like really nice people. I am probably wrong. They are going to do the right thing.”
But seriously, is that enough? Can I trust a few people with the power to shape the whole universe? Can I trust them enough to actually give them money? Can I trust them enough with my life until the end of the universe?
You can’t even tell me what “best” or “right” or “winning” stands for. How do I know that it can be or will be defined in a way that those labels will apply to me as well?
I have no idea what your plans are for the day when time runs out. I just hope that you are not going to hope for the best and run some not quite friendly AI that does really crappy things. I hope you consider the possibility of rather blowing everything up than risking even worse outcomes.
Hell no.
This is an open problem. See “How can we be sure a Friendly AI development team will be altruistic?” on my list of open problems.
Blowing everying up would be pretty bad. Bad enough to not encourage the possibility.