Jessica Rumbelow

• Jessica RumbelowFeb 5, 2023, 8:56 PM

  13 points

  0

  in reply to: lsusr’s comment on: SolidGoldMag­ikarp (plus, prompt gen­er­a­tion)

  What’s an SCP?

• Jessica RumbelowFeb 5, 2023, 7:24 PM

  3 points

  0

  in reply to: mic’s comment on: SolidGoldMag­ikarp (plus, prompt gen­er­a­tion)

  Not yet, but there’s no reason why it wouldn’t be possible. You can imagine microscope AI, for language models. It’s on our to-do list.

• Jessica RumbelowFeb 5, 2023, 7:24 PM

  2 points

  0

  in reply to: LawrenceC’s comment on: SolidGoldMag­ikarp (plus, prompt gen­er­a­tion)

  Good to know. Thanks!

• Jessica RumbelowFeb 5, 2023, 12:11 PM

  2 points

  0

  in reply to: Charlie Steiner’s comment on: SolidGoldMag­ikarp (plus, prompt gen­er­a­tion)

  Yep, aside from running forward prop n times to generate an output of length n, we can just optimise the mean probability of the target tokens at each position in the output—it’s already implemented in the code. Although, it takes way longer to find optimal completions.

• Jessica RumbelowFeb 4, 2023, 10:58 PM

  8 points

  0

  on: Adam Scherlis’s Shortform

  More detail on this phenomenon here: https://​​www.lesswrong.com/​​posts/​​aPeJE8bSo6rAFoLqg/​​solidgoldmagikarp-plus-prompt-generation

• Jessica RumbelowNov 26, 2022, 10:03 AM

  1 point

  0

  in reply to: Richard_Kennaway’s comment on: Guardian AI (Misal­igned sys­tems are all around us.)

  Yeah, I think it could be! I’m considering pursuing it after SERI-MATS. I’ll need a couple of cofounders.

Guardian AI (Misal­igned sys­tems are all around us.)

The Ground Truth Prob­lem (Or, Why Eval­u­at­ing In­ter­pretabil­ity Meth­ods Is Hard)

• Jessica RumbelowNov 11, 2022, 1:00 PM

  6 points

  1

  in reply to: Joseph Bloom’s comment on: Why I’m Work­ing On Model Ag­nos­tic Interpretability

  Hi Joseph! I’ll briefly address the saliency map concern here – it likely originates from this paper, which showed that some types of saliency mapping methods had no more explanatory power than edge detectors. It’s a great paper, and worth a read. The key thing to note is that this was only true of some gradient-based saliency mapping methods, which are, of course, model-specific. Gradients can be deceptive! Model agnostic, perturbation-based saliency mapping doesn’t suffer from the same kind of problems – see p.12 here.

Why I’m Work­ing On Model Ag­nos­tic Interpretability

• Jessica RumbelowOct 9, 2022, 8:15 PM

  3 points

  0

  on: [Cross­post] AlphaTen­sor, Taste, and the Scal­a­bil­ity of AI

  “being able to reorganise a question in the form of a model-appropriate game” seems like something we already have built a set of reasonable heuristics around—categorising different types of problems and their appropriate translations into ML-able tasks. There are well established ML approaches to, e.g. image captioning, time-series prediction, audio segmentation etc etc. is the bottleneck you’re concerned with the lack of breadth and granularity of these problem-sets, OP—and we can mark progress (to some extent) by the number of these problem sets we have robust ML translations for?