Matt Levinson

• Matt LevinsonMar 1, 2025, 12:19 AM

  1 point

  0

  on: Fuzzing LLMs some­times makes them re­veal their secrets

  This, along with @Andrew Mack’s MELBO and DCT work, is super cool and promising! One question, have you explored altering discovered vectors that make meaningful but non-gibberish changes to see if you can find something like a minimal viable direction? Perhaps something like taking successful vectors and then individually reoptimizing them turning down the L2 norm to see if some dimensions preferentially maintain their magnitude?

• Matt LevinsonJan 23, 2025, 11:46 PM

  12 points

  3

  in reply to: johnswentworth’s comment on: The Case Against AI Con­trol Research

  I don’t think your first paragraph applies to the first three bullets you listed.

  • Leaders don’t even bother to ask researchers to leverage the company’s current frontier model to help in what is hopefully the company-wide effort to reduce risk from the ASI model that’s coming? That’s a leadership problem, not a lack of technical understanding problem. I suppose if you imagine that a company could get to fine-grained mechanical understanding of everything their early AGI model does then they’d be more likely to ask because they think it will be easier/​faster? But we all know we’re almost certainly not going to have that understanding. Not asking would just be a leadership problem.

  • Leaders ask alignment team to safety-wash? Also a leadership problem.

  • Org can’t implement good alignment solutions their researchers devise? Again given that we all already know that we’re almost certainly not going to have comprehensive mechanical understanding of the early-AGI models, I don’t understand how shifts in the investment portfolio of technical AI safety research affects this? Still just seems a leadership problem unrelated to the percents next to each sub-field in the research investment portfolio.

  Which leads me to your last paragraph. Why write a whole post against AI control in this context? Is your claim that there are sub-fields of technical AI safety research that are significantly less threatened by your 7 bullets that offer plausible minimization of catastrophic AI risk? That we shouldn’t bother with technical AI safety research at all? Something else?

• Matt LevinsonJan 23, 2025, 8:17 PM

  14 points

  4

  on: The Case Against AI Con­trol Research

  As a community, I agree it’s important to have a continuous discussion on how to best shape research effort and public advocacy to maximally reduce X-risk. But this post feels off the mark to me. Consider your bullet list of sources of risk not alleviated by AI control. You proffer that your list makes up a much larger portion of the probability space than misalignment or deception. This is the pivot point in your decision to not support investing research resources in AI control.

  You list seven things. The first three aren’t addressable by any technical research or solution. Corporate leaders might be greedy, hubristic, and/​or reckless. Or human organizations might not be nimble enough to effect development and deployment of the maximum safety we are technically capable of. No safety research portfolio addresses those risks. The other four are potential failures by us as a technical community that apply broadly. If too high a percentage of the people in our space are bad statisticians, can’t think distributionally, are lazy or prideful, or don’t understand causal reasoning well enough, that will doom all potential directions of AI safety research, not just AI control.

  So in my estimation, if we grant that it is important to do AI safety research, we can ignore the entire list in the context of estimating the value of AI control versus other investment of research resources. You don’t argue against the proposition that controlling misalignment of early AGI is a plausible path to controlling ASI. You gesture at it to start but then don’t argue against the plausibility of the path of AI control of early AGI allowing collaboration to effect safe ASI through some variety of methods. You just list a bunch of—entirely plausible and I agree not unlikely—reasons why all attempts at preventing catastrophic risk might fail. I don’t understand why you single out AI control versus other technical AI safety sub-fields that are subject to all the same failure modes?

  What links here?

  • Noosphere89's comment on In­stru­men­tal Goals Are A Differ­ent And Friendlier Kind Of Thing Than Ter­mi­nal Goals by johnswentworth (Jan 25, 2025, 6:20 PM; 10 points)

• Matt LevinsonJan 13, 2025, 9:58 PM

  4 points

  0

  on: Scal­ing Sparse Fea­ture Cir­cuit Find­ing to Gemma 9B

  Very cool work! I think scalable circuit finding is an exciting and promising area that could get us to practically relevant oversight capabilities driven by mechint with not too too long a timeline!

  Did you think at all about ways to better capture interaction effects? I’ve thought about approaches similar to what you share here and really all that’s happening is a big lasso regression with the coefficients embedded in a functional form to make them “continuousified” indicator variables that contribute to the prediction part of the objective only by turning on or off the node they’re attached to. As is of course well known, lasso tends to select representative elements out of groups with strong interactions, or miss them entirely if the main effects are weak while only the interactions are meaningful. The stock answer in the classic regression context is to also include an L2 penalty to soften the edges of the L1 penalty contour, but this seems not a great answer in this context. We need the really strong sparsity bias of the solo L1 penalty in this context!

  I don’t mean this as a knock on this work! Really strong effort that would’ve just been bogged down by trying to tackle the covariance/​interaction problem on the first pass. I’m just wondering if you’ve had discussions or thoughts on that problem for future work in the journey of this research?

Ac­ti­va­tion Mag­ni­tudes Mat­ter On Their Own: In­sights from Lan­guage Model Distri­bu­tional Analysis

• Matt LevinsonDec 2, 2024, 9:13 PM

  1 point

  0

  on: Beyond Gaus­sian: Lan­guage Model Rep­re­sen­ta­tions and Distributions

  I’ve uploaded the code to github.

Beyond Gaus­sian: Lan­guage Model Rep­re­sen­ta­tions and Distributions

• Matt LevinsonSep 16, 2024, 10:27 PM

  1 point

  0

  on: How use­ful is mechanis­tic in­ter­pretabil­ity?

  I’m a new OpenPhil fellow for a mid-career transition—from other spaces in AI/​ML—into AI safety, with an interest in interpretability. Given my experience, I bias towards intuitively optimistic about mechanical interpretability in the sense of discovering representations and circuits and trying to make sense of them. But I’ve started my deep dive into the literature. I’d be really interested to hear from @Buck and @ryan_greenblatt and those who share their skepticism about what directions they prefer to invest for their own and their team’s research efforts!

  Out of the convo and the comments I got relying more on probes rather than dictionaries and circuits alone. But I feel pretty certain that’s not the complete picture! I came to this convo from the Causal Scrubbing thread which felt exciting to me and like a potential source of inspiration for a mini research project for my fellowship (6 months, including ramp up/​learning). I was a bit bummed to learn that the authors found the main benefit of that project to be informing them to abandon mech interp :-D

  On a related note, one of the other papers that put me on a path to this thread was this one on Causal Mediation. Fairly long ago at this point I had a phase of interest in Pearl’s causal theory and thought that paper was a nice example of thinking about what’s essentially ablation and activation patching from that point of view. Are there any folks who’ve taken a deeper stab at leveraging some of the more recent theoretical advances in graphical causal theory to do mech interp? Would super appreciate any pointers!