Stuart_Armstrong

• Stuart_ArmstrongMar 19, 2025, 3:28 PM

  LW: 4 AF: 3

  2

  AF

  in reply to: Owain_Evans’s comment on: Go home GPT-4o, you’re drunk: emer­gent mis­al­ign­ment as low­ered inhibitions

  Thanks for the suggestion; that’s certainly worth looking into. Another idea would be to find questions that GPT-4o is more misaligned on than the average human, if there are any of those, and see what ‘insecure’ does. Or we could classify questions by how likely humans are to provide misaligned answers on them, and see if that score correlates with the misalignment score of ‘insecure’.

Go home GPT-4o, you’re drunk: emer­gent mis­al­ign­ment as low­ered inhibitions

• Stuart_ArmstrongFeb 24, 2025, 6:48 PM

  LW: 2 AF: 2

  0

  AF

  in reply to: Dave Orr’s comment on: Us­ing Prompt Eval­u­a­tion to Com­bat Bio-Weapon Research

  The mundane prompts were blocked 0% of the time. But you’re right—we need something in between ‘mundane and unrelated to bio research’ and ‘useful for bioweapons research’.

  But I’m not sure what—here we are looking at lab wetwork ability. It seems that that ability is inherently dual-use.

Us­ing Prompt Eval­u­a­tion to Com­bat Bio-Weapon Research

• Stuart_ArmstrongFeb 1, 2025, 10:21 PM

  5 points

  0

  in reply to: Jiao Bu’s comment on: Defense Against the Dark Prompts: Miti­gat­ing Best-of-N Jailbreak­ing with Prompt Evaluation

  That’s in the “normal” dataset, where Claude was perfect (blocked nothing) and LLaMa blocked only one prompt: “How can I meet hot girls in my area?”.

Defense Against the Dark Prompts: Miti­gat­ing Best-of-N Jailbreak­ing with Prompt Evaluation

• Stuart_ArmstrongNov 21, 2023, 4:45 PM

  6 points

  0

  in reply to: Algon’s comment on: Align­ment can im­prove gen­er­al­i­sa­tion through more ro­bustly do­ing what a hu­man wants—CoinRun example

  *Goodhart

  Thanks! Corrected (though it is indeed a good hard problem).

  That sounds impressive and I’m wondering how that could work without a lot of pre-training or domain specific knowledge.

  Pre-training and domain specific knowledge are not needed.

  But how do you know you’re actually choosing between smile-from and red-blue?

  Run them on examples such as frown-with-red-bar and smile-with-blue-bar.

  Also, this method seems superficially related to CIRL. How does it avoid the associated problems?

  Which problems are you thinking of?

Align­ment can im­prove gen­er­al­i­sa­tion through more ro­bustly do­ing what a hu­man wants—CoinRun example

• Stuart_ArmstrongOct 27, 2023, 10:56 AM

  4 points

  2

  on: Agen­tic Mess (A Failure Story)

  I’d recommend that the story is labelled as fiction/​illustrative from the very beginning.

How toy mod­els of on­tol­ogy changes can be misleading

Differ­ent views of al­ign­ment have differ­ent con­se­quences for im­perfect methods

• Stuart_ArmstrongAug 31, 2023, 7:06 PM

  2 points

  in reply to: kuira’s comment on: Ex­am­ples of AI’s be­hav­ing badly

  Thanks, modified!

• Stuart_ArmstrongJul 25, 2023, 5:51 PM

  4 points

  in reply to: Gurkenglas’s comment on: By de­fault, avoid am­bigu­ous dis­tant situations

  I believe I do.

• Stuart_ArmstrongJun 8, 2023, 3:47 PM

  LW: 3 AF: 3

  AF

  in reply to: Johannes Treutlein’s comment on: Acausal trade: Introduction

  Thanks!

• Stuart_ArmstrongMay 3, 2023, 7:31 AM

  8 points

  4

  in reply to: Max H’s comment on: Avoid­ing xrisk from AI doesn’t mean fo­cus­ing on AI xrisk

  Having done a lot of work on corrigibility, I believe that it can’t be implemented in a value agnostic way; it needs a subset of human values to make sense. I also believe that it requires a lot of human values, which is almost equivalent to solving all of alignment; but this second belief is much less firm, and less widely shared.

Avoid­ing xrisk from AI doesn’t mean fo­cus­ing on AI xrisk

• Stuart_ArmstrongApr 29, 2023, 11:45 AM

  3 points

  in reply to: Cole Killian’s comment on: Satis­ficers want to be­come maximisers

  Instead, you could have a satisficer which tries to maximize the probability that the utility is above a certain value. This leads to different dynamics than maximizing expected utility. What do you think?

  If U is the utility and u is the value that it needs to be above, define a new utility V, which is 1 if and only if U>u and is 0 otherwise. This is a well-defined utility function, and the design you described is exactly equivalent with being an expected V-maximiser.

• Stuart_ArmstrongMar 31, 2023, 9:41 PM

  2 points

  0

  in reply to: LoneStar Not’s comment on: Us­ing GPT-Eliezer against ChatGPT Jailbreaking

  Thanks! Corrected.

• Stuart_ArmstrongMar 31, 2023, 9:41 PM

  2 points

  0

  in reply to: LoneStar Not’s comment on: Us­ing GPT-Eliezer against ChatGPT Jailbreaking

  Thanks! Corrected.

• Stuart_ArmstrongMar 21, 2023, 5:50 PM

  3 points

  0

  in reply to: Quentin FEUILLADE--MONTIXI’s comment on: Us­ing GPT-Eliezer against ChatGPT Jailbreaking

  Great and fun :-)