Maybe this is obvious but isn’t AI alignment only useful if you have access to the model? And aren’t well-funded governments the most likely to develop ‘dangerous’/strong AI, regardless of whether AI alignment “solutions” exist outside of the govt sphere?
Define access. It’s more like computer security. The closer you are to the bare metal, the more options you have. If you can manually change the weights of a NN, you can do more than if you can only change its training data. But both will influence the resulting model. It also depends on whether the AI can learn new stuff after deployment—if it can learn new tricks, that means that you can modify it remotely by crafting sneaky inputs.
The reason that well funded governments can develop interesting things is because they have enough resources to do so. If you give a private org enough resources, they can also do wonders. Like SpaceX. Of course you could say that they’re building on decades of hard work by NASA etc., but that goes to show that after a certain amount of external research is done to blaze the trail, then private companies can come in to exploit the new opportunities. This is sort of happening now with DeepMind, OpenAI etc.
That is what AI Governance focuses on. The pessimistic answer is that a single researcher can’t do anything. Though that really can be said about all aspects of government. Luckily most governments that count are at least sort of democratic and so have to take into account the people. Which implies that a researcher can’t do much, but a group of them might be able to. In other words—politics.
If you know how the AI works, you could work on coming up with malicious inputs, something like this, but more general. Though now we’re going deep into the weeds—most people here are more focused on how to make the AI not want to kill us, rather than not letting it be used to kill us.
Maybe this is obvious but isn’t AI alignment only useful if you have access to the model? And aren’t well-funded governments the most likely to develop ‘dangerous’/strong AI, regardless of whether AI alignment “solutions” exist outside of the govt sphere?
Define access. It’s more like computer security. The closer you are to the bare metal, the more options you have. If you can manually change the weights of a NN, you can do more than if you can only change its training data. But both will influence the resulting model. It also depends on whether the AI can learn new stuff after deployment—if it can learn new tricks, that means that you can modify it remotely by crafting sneaky inputs.
The reason that well funded governments can develop interesting things is because they have enough resources to do so. If you give a private org enough resources, they can also do wonders. Like SpaceX. Of course you could say that they’re building on decades of hard work by NASA etc., but that goes to show that after a certain amount of external research is done to blaze the trail, then private companies can come in to exploit the new opportunities. This is sort of happening now with DeepMind, OpenAI etc.
What can a researcher even do against a government that’s using the AI to fulfill their own unscrupulous/not-optimizing-for-societal-good goals?
That is what AI Governance focuses on. The pessimistic answer is that a single researcher can’t do anything. Though that really can be said about all aspects of government. Luckily most governments that count are at least sort of democratic and so have to take into account the people. Which implies that a researcher can’t do much, but a group of them might be able to. In other words—politics.
If you know how the AI works, you could work on coming up with malicious inputs, something like this, but more general. Though now we’re going deep into the weeds—most people here are more focused on how to make the AI not want to kill us, rather than not letting it be used to kill us.