Stuart_Armstrong

• Stuart_Armstrong 21 Nov 2023 16:45 UTC

  6 points

  0

  in reply to: Algon’s comment on: Align­ment can im­prove gen­er­al­i­sa­tion through more ro­bustly do­ing what a hu­man wants—CoinRun example

  *Goodhart

  Thanks! Corrected (though it is indeed a good hard problem).

  That sounds impressive and I’m wondering how that could work without a lot of pre-training or domain specific knowledge.

  Pre-training and domain specific knowledge are not needed.

  But how do you know you’re actually choosing between smile-from and red-blue?

  Run them on examples such as frown-with-red-bar and smile-with-blue-bar.

  Also, this method seems superficially related to CIRL. How does it avoid the associated problems?

  Which problems are you thinking of?

Align­ment can im­prove gen­er­al­i­sa­tion through more ro­bustly do­ing what a hu­man wants—CoinRun example

• Stuart_Armstrong 27 Oct 2023 10:56 UTC

  4 points

  2

  on: Agen­tic Mess (A Failure Story)

  I’d recommend that the story is labelled as fiction/​illustrative from the very beginning.

How toy mod­els of on­tol­ogy changes can be misleading

Differ­ent views of al­ign­ment have differ­ent con­se­quences for im­perfect methods

• Stuart_Armstrong 31 Aug 2023 19:06 UTC

  2 points

  in reply to: kuira’s comment on: Ex­am­ples of AI’s be­hav­ing badly

  Thanks, modified!

• Stuart_Armstrong 25 Jul 2023 17:51 UTC

  4 points

  in reply to: Gurkenglas’s comment on: By de­fault, avoid am­bigu­ous dis­tant situations

  I believe I do.

• Stuart_Armstrong 8 Jun 2023 15:47 UTC

  LW: 3 AF: 3

  AF

  in reply to: Johannes Treutlein’s comment on: Acausal trade: Introduction

  Thanks!

• Stuart_Armstrong 3 May 2023 7:31 UTC

  8 points

  4

  in reply to: Max H’s comment on: Avoid­ing xrisk from AI doesn’t mean fo­cus­ing on AI xrisk

  Having done a lot of work on corrigibility, I believe that it can’t be implemented in a value agnostic way; it needs a subset of human values to make sense. I also believe that it requires a lot of human values, which is almost equivalent to solving all of alignment; but this second belief is much less firm, and less widely shared.

Avoid­ing xrisk from AI doesn’t mean fo­cus­ing on AI xrisk

• Stuart_Armstrong 29 Apr 2023 11:45 UTC

  3 points

  in reply to: Cole Killian’s comment on: Satis­ficers want to be­come maximisers

  Instead, you could have a satisficer which tries to maximize the probability that the utility is above a certain value. This leads to different dynamics than maximizing expected utility. What do you think?

  If U is the utility and u is the value that it needs to be above, define a new utility V, which is 1 if and only if U>u and is 0 otherwise. This is a well-defined utility function, and the design you described is exactly equivalent with being an expected V-maximiser.

• Stuart_Armstrong 31 Mar 2023 21:41 UTC

  2 points

  0

  in reply to: LoneStar Not’s comment on: Us­ing GPT-Eliezer against ChatGPT Jailbreaking

  Thanks! Corrected.

• Stuart_Armstrong 31 Mar 2023 21:41 UTC

  2 points

  0

  in reply to: LoneStar Not’s comment on: Us­ing GPT-Eliezer against ChatGPT Jailbreaking

  Thanks! Corrected.

• Stuart_Armstrong 21 Mar 2023 17:50 UTC

  3 points

  0

  in reply to: Quentin FEUILLADE--MONTIXI’s comment on: Us­ing GPT-Eliezer against ChatGPT Jailbreaking

  Great and fun :-)

What is a defi­ni­tion, how can it be ex­trap­o­lated?

You’re not a simu­la­tion, ’cause you’re hallucinating

• Stuart_Armstrong 10 Feb 2023 17:35 UTC

  2 points

  0

  in reply to: Roman Leventov’s comment on: Refin­ing the Sharp Left Turn threat model, part 2: ap­ply­ing al­ign­ment techniques

  Another way of saying this is that inner alignment is more important than outer alignment.

  Interesting. My intuition is the inner alignment has nothing to do with this problem. It seems that different people view the inner vs outer alignment distinction in different ways.

• Stuart_Armstrong 8 Feb 2023 11:46 UTC

  2 points

  0

  in reply to: mwatkins’s comment on: SolidGoldMag­ikarp (plus, prompt gen­er­a­tion)

  Thanks! Yes, this is some weird behaviour.

  Keep me posted on any updates!

• Stuart_Armstrong 6 Feb 2023 13:10 UTC

  LW: 3 AF: 2

  0

  AF

  on: SolidGoldMag­ikarp (plus, prompt gen­er­a­tion)

  As we discussed, I feel that the tokens were added for some reason but then not trained on; hence why they are close to the origin, and why the algorithm goes wrong on them, because it just isn’t trained on them at all.

  Good work on this post.

• Stuart_Armstrong 5 Feb 2023 19:39 UTC

  LW: 8 AF: 4

  3

  AF

  on: Refin­ing the Sharp Left Turn threat model, part 2: ap­ply­ing al­ign­ment techniques

  I’ll be very boring and predictable and make the usual model splintering/​value extrapolation point here :-)

  Namely that I don’t think we can talk sensibly about an AI having “beneficial goal-directedness” without situational awareness. For instance, it’s of little use to have an AI with the goal of “ensuring human flourishing” if it doesn’t understand the meaning of flourishing or human. And, without situational awareness, it can’t understand either; at best we could have some proxy or pointer towards these key concepts.

  The key challenge seems to be to get the AI to generalise properly; even initially poor goals can work if generalised well. For instance, a money-maximising trade-bot AI could be perfectly safe if it notices that money, in its initial setting, is just a proxy for humans being able to satisfy their preferences.

  So I’d be focusing on “do the goals stay safe as the AI gains situational awareness?”, rather than “are the goals safe before the AI gains situational awareness?”