AI alignment is more often than not assumed to be with some abstract “human values” which are ought to be collected from the general public and aggregated with some clever, yet undeveloped algorithms. I don’t think anyone in AI alignment, once “solved” this problem, would hand the technology over to POTUS or any other powerful leader or a clique.
My impression is that MIRI would prefer some sort of TESCREAList future, with emulated minds colonizing all of space, or something like that. This is probably something most people would be afraid of. MIRI isn’t the only AI alignment research org, but I don’t really see other rationalist alignment researchers opposing this desire, I think because they’re all TESCREALists?
Of course MIRI doesn’t seem to be on track towards grabbing control over the world themselves to implement TESCREALism, so more realistically MIRI might produce something that the leading tech companies like OpenAI can incorporate in their work. But “tech companies” were one of the original sides mentioned in the OP that “realists” were worried about.
And furthermore, the tech companies have recently decided that they don’t know how to control the AI and begged the US government to take control. Rationalists don’t really expect the government to do well with this on its own, but I think there is hope that AI safety researchers could produce some tools that the US government could make mandates about using? But that would also mean that “doomers” are giving the US government a lot of power?
And furthermore, the tech companies have recently decided that they don’t know how to control the AI and begged the US government to take control. Rationalists don’t really expect the government to do well with this on its own, but I think there is hope that AI safety researchers could produce some tools that the US government could make mandates about using? But that would also mean that “doomers” are giving the US government a lot of power?
I think you mischaracterise what’s happening a lot. Tech companies don’t “beg US government to control AI”, they beg it to regulate the industry. That’s a very big difference. They didn’t yet say anything about the control of AI, once it’s build, and (presumably) “aligned”, apart from well-sounding for PR phrases like “we will increasingly involve input from more people about where do we take AI”. In reality, they just don’t know yet who and how should “control” AI (if anyone), and align with whose “values”. They hope that the “alignment MVP”, a.k.a. AGI which is an aligned AI safety and alignment scientist, will actually tell them the “right” answers to these questions (as per OpenAI’s “superalignment” agenda).
AI alignment is more often than not assumed to be with some abstract “human values” which are ought to be collected from the general public and aggregated with some clever, yet undeveloped algorithms. I don’t think anyone in AI alignment, once “solved” this problem, would hand the technology over to POTUS or any other powerful leader or a clique.
Maybe sometimes? Mostly I disagree.
My impression is that MIRI would prefer some sort of TESCREAList future, with emulated minds colonizing all of space, or something like that. This is probably something most people would be afraid of. MIRI isn’t the only AI alignment research org, but I don’t really see other rationalist alignment researchers opposing this desire, I think because they’re all TESCREALists?
Of course MIRI doesn’t seem to be on track towards grabbing control over the world themselves to implement TESCREALism, so more realistically MIRI might produce something that the leading tech companies like OpenAI can incorporate in their work. But “tech companies” were one of the original sides mentioned in the OP that “realists” were worried about.
And furthermore, the tech companies have recently decided that they don’t know how to control the AI and begged the US government to take control. Rationalists don’t really expect the government to do well with this on its own, but I think there is hope that AI safety researchers could produce some tools that the US government could make mandates about using? But that would also mean that “doomers” are giving the US government a lot of power?
I think you mischaracterise what’s happening a lot. Tech companies don’t “beg US government to control AI”, they beg it to regulate the industry. That’s a very big difference. They didn’t yet say anything about the control of AI, once it’s build, and (presumably) “aligned”, apart from well-sounding for PR phrases like “we will increasingly involve input from more people about where do we take AI”. In reality, they just don’t know yet who and how should “control” AI (if anyone), and align with whose “values”. They hope that the “alignment MVP”, a.k.a. AGI which is an aligned AI safety and alignment scientist, will actually tell them the “right” answers to these questions (as per OpenAI’s “superalignment” agenda).