niki.h

Does ro­bust­ness im­prove with scale?

• niki.h 2 Nov 2023 17:59 UTC

  5 points

  2

  on: Say­ing the quiet part out loud: trad­ing off x-risk for per­sonal immortality

  Based on personal experience, you are definitely not the only one thinking about that Statement.

• niki.h 2 Nov 2023 17:29 UTC

  1 point

  0

  in reply to: philip_b’s comment on: Wire­head­ing and mis­al­ign­ment by com­po­si­tion on NetHack

  Based on my understanding from talking with the author, it is the former. The language model is simply used to provide a shaping reward based on the text outputs that the game shows after some actions; it’s the RL optimization that learns the weird hallucination strategy, and the reason it’s able to do it is because its capabilities in general are improved thanks to the shaping reward.