Hypothesis: Safety techniques show promise but provide no evidence that we can achieve AI agents that are fundamentally different than humans in respect to safety and alignment. In my opinion, safe and aligned AI can be as safe and aligned as humans, who can:
be moral and safe but can still be misused and cause harm.
be moral and aligned but can still deceive to achieve a goal or miss some variables and make fatal mistakes
Conclusion: Emphasize the limits of safe and aligned AI while making policies for regulation. Keep stressing the question if we are prepared to face the repercussions of the likelihood of safe and aligned AI causing harmat that scale that fast
Why this post?
This is an opinion piece. I have tried not to look into the theoretical works on AI safety yet so as to develop my own conclusions and ideas on AI safety.
Even if this post is written with the risk on not adding anything new, I believe it serves as a documentation of independent opinions on AI risks and AI safety approaches developed purely based on readings about several AI safety techniques and avoiding influence from different schools of thought in this area.
I have worked as a deep learning researcher at a University for two years after a PhD in Mathematics and recently attended AI safety fundamentals by BlueDot.
Best-case-scenario AI
I will avoid the discussion on ASI. I believe that some capability researchers and policy professionals undermine and downplay the discussions on safety on the mention of ASI. For the sake of discussion, I will agree with them on ASI and focus on much simpler projection of AI:
AI models are getting bigger and advanced
AI safety and AI alignment research is enabling useful and safer AI agents.
So, here I focus on the increasingly capable AI in the immediate future in its safest and most alignedform and refer itas the best-case-scenario AI.
In my humble opinion, in this projection scenario, absolutely safe and absolutely aligned AI agents do not exist.
Limits of the best-case-scenario AI
In terms of policy, we would be better off if we assume the following for the best-case-scenario AI:
It will enable intelligent agents of human or better than human level intelligence that would be part of our society which can only be aligned as good as humans can be aligned.
They will always be prone to be misused by bad actors by persuasion or force; and similar to humans, they might still deceive, lie, and do harm to fulfill certain goals assigned to them.
We can only have control over them as much we have on humans. For example, we will be able to put them under
- lie detectors (Representation Engineering),
- scrutinize them (Mechanistic Interpretability),
- question them (Chain-of-thought),
and more but without having absolute control over their actions. And thus have to account for the risk of failure.
We will be able to kill specific instances of them but we still don’t understand how they will be propagating.
Why should we assume so?
These are safer assumptions when we don’t see a really strong evidence suggesting otherwise. Ongoing work on safety and alignment, although very important and promising, gets more attention in mainstream narrative. This leads to an overestimation of its potential and a discussion on its limits, which is very important for policy making, generally tends to get ignored.
A bit more detail
Hypothesis 1: There will be no safe AI same as there are no safe humans
Claim: Safe AI can always be persuaded to do harm, same as humans.
What is true nevertheless?
AI can be made safer and harder to manipulate.
Advanced AI will enable us to make safer AI. But we need to emphasize that safer is relative to what we have now but at the same time better AI will also be more dangerous than we have now.
At every stage of its capabilities it will come with its good, its bad, and its ugly.
Why it matters?
Safe AI systems fine-tuned with safety techniques such as RLHF, Constitutional AI, debate, etc., will always be prone to Jailbreaking.
If we evaluate the AI capabilities based on what can be elicited from it based on the carefully crafted prompt, we should do the same for safety certifications.
Jailbreak is a carefully crafted prompt.And once found can be used untraced for long to derive dangerous capabilities.
Impact
AI companies should accept that jailbreaks will always exist. And in the cases where they want to ensure that their services are not being misused, they should be partially monitoring the responses of AI and not just rely on AI score of harmlessness based on red teaming. We need to account for the scale of risks for misuse of AI at every stage of its capability.
Hypothesis 2: There is no AI in AI Alignment problem
Alignment is a problem independent of AI that exists because it is hard to specify things.
Optimizing for anything that is general enough runs the risk of not being a useful instruction eg. Task prompt: Do what is good for the company. Response: AI concludes it is better to do nothing.
Optimizing for anything specific runs the risk of compromising variables that are not mentioned. Task prompt: Increase the clicks on this website. Response: AI posts false clickbait, which increases the number of clicks but defeats the goal of the task by making clients angry.
Claim: No evidence that aligned AI can be fundamentally better than an aligned intern.
Any issue in AI alignment is parallel to a smart newly hired intern in a company.
We are able to work with human interns by more or less giving specific task instructions by relying on a shared value system fine-tuned based on the company and learned socially, which we believe the intern will apply when facing a deliberation between different compromises.
We can use various safety techniques to add similar value systems in AI e.g. Constitutional AI. Still, there is no reason to believe that it will result in something fundamentally different than humans when it comes to understanding the intent of tasks.
Why it matters?
AI safety policy and regulation conversation gets muddled by an overestimation of the outcomes that can be achieved by AI alignment.
Impact
In policy discussions, we need to emphasize the limits of alignment in Aligned (best-case-scenario) AI and talk about the risks involved with the likelihood of
mistakes due to misinterpretation of the intent, and
harm due to deceptive behaviour
at the scale powerful aligned agents can cause.
This post adopts a simplified projection of AI for the purpose of policy discussions. It is assumed as a better course of immediate action whenever it is harder to substantiate the impact of the jump in intelligence on the emergence of ASI. Please critique and suggest links to articles and organizations that are working or want to work with a similar approach.
Limits of safe and aligned AI
TLDR:
Hypothesis: Safety techniques show promise but provide no evidence that we can achieve AI agents that are fundamentally different than humans in respect to safety and alignment. In my opinion, safe and aligned AI can be as safe and aligned as humans, who can:
be moral and safe but can still be misused and cause harm.
be moral and aligned but can still deceive to achieve a goal or miss some variables and make fatal mistakes
Conclusion: Emphasize the limits of safe and aligned AI while making policies for regulation. Keep stressing the question if we are prepared to face the repercussions of the likelihood of safe and aligned AI causing harm at that scale that fast
Why this post?
This is an opinion piece. I have tried not to look into the theoretical works on AI safety yet so as to develop my own conclusions and ideas on AI safety.
Even if this post is written with the risk on not adding anything new, I believe it serves as a documentation of independent opinions on AI risks and AI safety approaches developed purely based on readings about several AI safety techniques and avoiding influence from different schools of thought in this area.
I have worked as a deep learning researcher at a University for two years after a PhD in Mathematics and recently attended AI safety fundamentals by BlueDot.
Best-case-scenario AI
I will avoid the discussion on ASI. I believe that some capability researchers and policy professionals undermine and downplay the discussions on safety on the mention of ASI. For the sake of discussion, I will agree with them on ASI and focus on much simpler projection of AI:
AI models are getting bigger and advanced
AI safety and AI alignment research is enabling useful and safer AI agents.
So, here I focus on the increasingly capable AI in the immediate future in its safest and most aligned form and refer it as the best-case-scenario AI.
In my humble opinion, in this projection scenario, absolutely safe and absolutely aligned AI agents do not exist.
Limits of the best-case-scenario AI
In terms of policy, we would be better off if we assume the following for the best-case-scenario AI:
It will enable intelligent agents of human or better than human level intelligence that would be part of our society which can only be aligned as good as humans can be aligned.
They will always be prone to be misused by bad actors by persuasion or force; and similar to humans, they might still deceive, lie, and do harm to fulfill certain goals assigned to them.
We can only have control over them as much we have on humans. For example, we will be able to put them under
- lie detectors (Representation Engineering),
- scrutinize them (Mechanistic Interpretability),
- question them (Chain-of-thought),
and more but without having absolute control over their actions. And thus have to account for the risk of failure.
We will be able to kill specific instances of them but we still don’t understand how they will be propagating.
Why should we assume so?
These are safer assumptions when we don’t see a really strong evidence suggesting otherwise. Ongoing work on safety and alignment, although very important and promising, gets more attention in mainstream narrative. This leads to an overestimation of its potential and a discussion on its limits, which is very important for policy making, generally tends to get ignored.
A bit more detail
Hypothesis 1: There will be no safe AI same as there are no safe humans
Claim: Safe AI can always be persuaded to do harm, same as humans.
What is true nevertheless?
AI can be made safer and harder to manipulate.
Advanced AI will enable us to make safer AI. But we need to emphasize that safer is relative to what we have now but at the same time better AI will also be more dangerous than we have now.
At every stage of its capabilities it will come with its good, its bad, and its ugly.
Why it matters?
Safe AI systems fine-tuned with safety techniques such as RLHF, Constitutional AI, debate, etc., will always be prone to Jailbreaking.
Jailbreak is a carefully crafted prompt. And once found can be used untraced for long to derive dangerous capabilities.
Impact
AI companies should accept that jailbreaks will always exist. And in the cases where they want to ensure that their services are not being misused, they should be partially monitoring the responses of AI and not just rely on AI score of harmlessness based on red teaming. We need to account for the scale of risks for misuse of AI at every stage of its capability.
Hypothesis 2: There is no AI in AI Alignment problem
Alignment is a problem independent of AI that exists because it is hard to specify things.
Optimizing for anything that is general enough runs the risk of not being a useful instruction eg. Task prompt: Do what is good for the company. Response: AI concludes it is better to do nothing.
Optimizing for anything specific runs the risk of compromising variables that are not mentioned. Task prompt: Increase the clicks on this website. Response: AI posts false clickbait, which increases the number of clicks but defeats the goal of the task by making clients angry.
Claim: No evidence that aligned AI can be fundamentally better than an aligned intern.
Any issue in AI alignment is parallel to a smart newly hired intern in a company.
We can use various safety techniques to add similar value systems in AI e.g. Constitutional AI. Still, there is no reason to believe that it will result in something fundamentally different than humans when it comes to understanding the intent of tasks.
Why it matters?
AI safety policy and regulation conversation gets muddled by an overestimation of the outcomes that can be achieved by AI alignment.
Impact
In policy discussions, we need to emphasize the limits of alignment in Aligned (best-case-scenario) AI and talk about the risks involved with the likelihood of
mistakes due to misinterpretation of the intent, and
harm due to deceptive behaviour
at the scale powerful aligned agents can cause.
This post adopts a simplified projection of AI for the purpose of policy discussions. It is assumed as a better course of immediate action whenever it is harder to substantiate the impact of the jump in intelligence on the emergence of ASI. Please critique and suggest links to articles and organizations that are working or want to work with a similar approach.