You voiced the same concern I have, I am very grateful for this!
Yes, politics and regulations are not the focus of most LessWrongers. But we wouldn’t have the advances that we’re seeing without the contributions of those who care.
For example: How is it possible that, for the first time, “alignment with human intent” has been explicitly mentioned by a law, framed as a key concern for regulation of systemic risks, and most people in this community do not know?
This is a massive victory for the AI Safety / alignment community.
European Artificial Intelligence Act, Recital 110
The full text of the recital is very long. It describes what the law understand “systemic risks” to be, and how they can impact society.
Here is where alignment (with the meaning that this community gives it) is mentioned:
“International approaches have so far identified the need to pay attention to risks from potential intentional misuse or unintended issues of control relating to alignment with human intent”.
The EU AI Act also mentions alignment as part of the Technical documentation that AI developers must make publicly available.
I am particularly concerned about your point “3. We have evidence that the governance naysayers are badly calibrated”.
Last month, I attended a meeting by an European institution tasked with drafting the General Purpose AI Codes of Practice (the documents that companies like OpenAI can use to “prove compliance” with the law).
As the document puts a lot of emphasis on transparency, I raised a question to the panel about incentivizing mechanistic interpretability.
The majority of the experts didn’t know what I was talking about, and had never heard of such thing as “mechanistic interpretability”...
This was a personal wake up call for me, as a Lawyer and AI Safety researcher.
@Severin T. Seehrich, @Benjamin Schmidt- feel free to connect separately if you want! I am creating resources for AI Governance professionals to gain better understanding of AI Safety, and its potential to inform policymakers and improve the regulatory landscape.
You voiced the same concern I have, I am very grateful for this!
Yes, politics and regulations are not the focus of most LessWrongers. But we wouldn’t have the advances that we’re seeing without the contributions of those who care.
For example: How is it possible that, for the first time, “alignment with human intent” has been explicitly mentioned by a law, framed as a key concern for regulation of systemic risks, and most people in this community do not know?
This is a massive victory for the AI Safety / alignment community.
European Artificial Intelligence Act, Recital 110
The full text of the recital is very long. It describes what the law understand “systemic risks” to be, and how they can impact society.
See the full text here:https://artificialintelligenceact.eu/recital/110/
Here is where alignment (with the meaning that this community gives it) is mentioned:
“International approaches have so far identified the need to pay attention to risks from potential intentional misuse or unintended issues of control relating to alignment with human intent”.
The EU AI Act also mentions alignment as part of the Technical documentation that AI developers must make publicly available.
I am particularly concerned about your point “3. We have evidence that the governance naysayers are badly calibrated”.
Last month, I attended a meeting by an European institution tasked with drafting the General Purpose AI Codes of Practice (the documents that companies like OpenAI can use to “prove compliance” with the law).
As the document puts a lot of emphasis on transparency, I raised a question to the panel about incentivizing mechanistic interpretability.
The majority of the experts didn’t know what I was talking about, and had never heard of such thing as “mechanistic interpretability”...
This was a personal wake up call for me, as a Lawyer and AI Safety researcher.
@Severin T. Seehrich, @Benjamin Schmidt- feel free to connect separately if you want! I am creating resources for AI Governance professionals to gain better understanding of AI Safety, and its potential to inform policymakers and improve the regulatory landscape.