There may be numerous framings of what went wrong or what might be addressed to fix it, details sufficient to give real predictive power will probably be complicate and it’s a good bet that however interested “the powers that be” are in GOF, they’re probably much MUCH more interested in AI development. So there can be even more resources to spin the story in favor of forestalling any pressure that might build to regulate.
My main thesis regarding how a non-existential AI disaster would happen in practice is (and I don’t think this would happen), Google or Facebook or some other large tech company publicly releases an agent that’s intelligent enough to be middling at wargames but not enough to do things like creative ML research, and people put it in one or more of IOT devices/critical infrastructure/ military equipment. Surprise: it has a bad value function and/or edge case behavior, and a group of agents end up deliberately and publicly defecting and successfully killing large numbers of people.
In this scenario, it would be extremely obvious that the party responsible for marketing and selling the AI was FaceGoog, and no matter what the Powers That Be wanted, the grieving would be directing their anger towards those engineers. Politicians wouldn’t individually give much of a shit about the well being of The Machine and instead be racing to see who could make the most visible condemnations of Big Tech and argue over which party predicted this would happen all along. Journalists would do what they always do and spin the story according to their individual political ideologies and not according to some institutional incentives, which would be more about painting their political opponents as Big Tech supporters than instrumentally supporting the engineers. Whatever company was responsible for the problem would, at a minimum, shutter all AI research. Congress would pass some laws written by their lobbyist consultants, of whom who knows, maybe even one or two could even be said to be “alignment people”, and there’s a new body of oversight analogous to the FDA for biotech companies.
And I appreciate the viewpoint that this is either just one timeline, or relies on premises that might be untrue, but in my head at least it just seems like it falls into place without making many critical assumptions.
My main thesis regarding how a non-existential AI disaster would happen in practice is (and I don’t think this would happen), Google or Facebook or some other large tech company publicly releases an agent that’s intelligent enough to be middling at wargames but not enough to do things like creative ML research, and people put it in one or more of IOT devices/critical infrastructure/ military equipment. Surprise: it has a bad value function and/or edge case behavior, and a group of agents end up deliberately and publicly defecting and successfully killing large numbers of people.
In this scenario, it would be extremely obvious that the party responsible for marketing and selling the AI was FaceGoog, and no matter what the Powers That Be wanted, the grieving would be directing their anger towards those engineers. Politicians wouldn’t individually give much of a shit about the well being of The Machine and instead be racing to see who could make the most visible condemnations of Big Tech and argue over which party predicted this would happen all along. Journalists would do what they always do and spin the story according to their individual political ideologies and not according to some institutional incentives, which would be more about painting their political opponents as Big Tech supporters than instrumentally supporting the engineers. Whatever company was responsible for the problem would, at a minimum, shutter all AI research. Congress would pass some laws written by their lobbyist consultants, of whom who knows, maybe even one or two could even be said to be “alignment people”, and there’s a new body of oversight analogous to the FDA for biotech companies.
And I appreciate the viewpoint that this is either just one timeline, or relies on premises that might be untrue, but in my head at least it just seems like it falls into place without making many critical assumptions.