I did wonder if this was AI written before seeing the comment thread. It takes a lot of effort for a human to write like an AI. Upvoted for effort.
I think this also missed the mark with the LW audience because it is about AI safety, which is largely separate from AGI alignment. LW is mostly focused on the second. It’s addressing future systems that have their own goals, whereas AI safety addresses tool systems that don’t really have their own goals. Those issues are important; but around here we tend to worry that we’ll all die when a new generation of systems with goals are introduced, so we mostly focus on those. There is a lot of overlap between the two, but that’s mostly in technical implementations rather than choosing desired behavior.
The main point I am trying to make is that AGI risks cannot be deduced or theorised solely in abstract terms. They must be understood through rigorous empirical research on complex systems. If you view AI as an agent in the world, then it functions as a complex intervention. It may or may not act as intended by its designer, it may or may not deliver preferred outcomes, and it may or may not be acceptable to the user. There are ways to estimate uncertainty in each of these parameters through empirical research. Actually, there are degrees to which it acts as intended, degrees to which it is acceptable, and so on. This calls for careful empirical research and system-level understanding.
I write academic papers in healthcare, psychology, and epidemiology for peer-review. I don’t write blog posts every day, so thank you for your patience with this particular style, which was devised for guidelines and frameworks.
Thank you for sharing your thoughts on AI alignment, AI safety, and imminent threats. I posted this essay to demonstrate how public health guidelines and system thinking can be useful in preventing harm, inequality, and avoiding unforeseen negative outcomes in general. I wanted the LessWrong audience to gain perspectives from other fields that have been addressing rapidly emerging innovations—along with their benefits and harms—for centuries, with the aim of minimising risk and maximising benefit, keeping the wider public in mind.
I am aware of the narrative around the ‘paperclip maximiser’ threat. However, I believe it is important to recognise that the risks AI brings should not be viewed in the context of a single threat, a single bias, or one path to extinction. AI is a complex system, used in a complex setting—the social structure. It should be studied with due rigour, with a focus on understanding its complexity.
If you can suggest literature on AGI alignment that recognises the complexity of the issue and applies systems thinking to the problem, I would be grateful.
I did wonder if this was AI written before seeing the comment thread. It takes a lot of effort for a human to write like an AI. Upvoted for effort.
I think this also missed the mark with the LW audience because it is about AI safety, which is largely separate from AGI alignment. LW is mostly focused on the second. It’s addressing future systems that have their own goals, whereas AI safety addresses tool systems that don’t really have their own goals. Those issues are important; but around here we tend to worry that we’ll all die when a new generation of systems with goals are introduced, so we mostly focus on those. There is a lot of overlap between the two, but that’s mostly in technical implementations rather than choosing desired behavior.
The main point I am trying to make is that AGI risks cannot be deduced or theorised solely in abstract terms. They must be understood through rigorous empirical research on complex systems. If you view AI as an agent in the world, then it functions as a complex intervention. It may or may not act as intended by its designer, it may or may not deliver preferred outcomes, and it may or may not be acceptable to the user. There are ways to estimate uncertainty in each of these parameters through empirical research. Actually, there are degrees to which it acts as intended, degrees to which it is acceptable, and so on. This calls for careful empirical research and system-level understanding.
I write academic papers in healthcare, psychology, and epidemiology for peer-review. I don’t write blog posts every day, so thank you for your patience with this particular style, which was devised for guidelines and frameworks.
Thank you for sharing your thoughts on AI alignment, AI safety, and imminent threats. I posted this essay to demonstrate how public health guidelines and system thinking can be useful in preventing harm, inequality, and avoiding unforeseen negative outcomes in general. I wanted the LessWrong audience to gain perspectives from other fields that have been addressing rapidly emerging innovations—along with their benefits and harms—for centuries, with the aim of minimising risk and maximising benefit, keeping the wider public in mind.
I am aware of the narrative around the ‘paperclip maximiser’ threat. However, I believe it is important to recognise that the risks AI brings should not be viewed in the context of a single threat, a single bias, or one path to extinction. AI is a complex system, used in a complex setting—the social structure. It should be studied with due rigour, with a focus on understanding its complexity.
If you can suggest literature on AGI alignment that recognises the complexity of the issue and applies systems thinking to the problem, I would be grateful.