Michael Kratsios strikes me as the person most worth keeping an eye on in terms of Trump’s potential AI policy
Caleb W
Karma: 49
A quick nitpick:
You say:The executive order was intended to limit various risks from AI… none of which were at all related to human extinction, except maybe bioweapons.
But note this from the EO:
(k) The term “dual-use foundation model” means an AI model that is trained on broad data; generally uses self-supervision; contains at least tens of billions of parameters; is applicable across a wide range of contexts; and that exhibits, or could be easily modified to exhibit, high levels of performance at tasks that pose a serious risk to security, national economic security, national public health or safety, or any combination of those matters, such as by:
...(iii) permitting the evasion of human control or oversight through means of deception or obfuscation.
I’m pretty okay with this having been done—
seems broadly consistent with cyber disclosure norms
- we know that there are people who have disputes with this community and a track record of looking to de-anonymize accounts, so relying on security through obscurity doesn’t seem that compelling
- seems reasonable to view this post as helpful to would-be anonymous posters—better to find out about the possibility this way than to be surprised by someone publicly attempting to deanonymize you in future, and the suggestion of using an LLM to change the style seems constructive
Given that no-one’s posted a comment in the affirmative yet:
I’d guess that more US national security engagement with AI risk is good. In rough order why:
I think the deployment problem is a key challenge, and an optimal strategy for addressing this challenge will have elements of transnational competition, information security, and enforcement that benefit from or require the national security apparatus.
As OP points out, there’s some chance that the US government/military ends up as a key player advancing capabilities, so it’s good for them to be mindful of the risks.
As OP points out, if funding for large alignment projects seems promising, places like DTRA have large budgets and a strong track record of research funding.
I agree that there are risks with communicating AI risk concepts in a way that poisons the well, lacks fidelity, gets distorted, or fails to cross inferential distances, but these seem like things to manage and mitigate rather than give up on. Illustratively, I’d be excited about bureaucrats, analysts and program managers reading things like Alignment Problem from a Deep Learning Perspective, Unsolved Problems in ML Safety, or CSET’s Key Concepts in AI Safety series; and developing frameworks and triggers to consider whether and when cutting-edge AI systems merit regulatory attention as dual use and/or high-risk systems a la the nuclear sector. (I include these examples as things that seem directionally good to me off the top of my head, but I’m not claiming they’re the most promising things to push on in this space).
This does seem like an important issue to consider, but my guess is it probably shouldn’t be a crux for answering OP’s question (or at least, further explanation is needed for why it might be)? Putting aside concerns about flawed pursuit of a given goal, it would be surprising if the benefits of caring about a goal were outweighed by second order harms from competitors extracting concessions.
Not at the scale you’re suggesting, but relevant: https://futureoflife.org/recent-news/50000-award-to-stanislav-petrov-for-helping-avert-wwiii-but-us-denies-visa/