Yes, I agree. It’s very annoying for general epistemics (though obviously pragmatically useful to me in various ways if people respect my opinion)
Though, to be clear, my main goal in writing this post was not to request that people defer less to me specifically, but more to make the general point about please defer more intelligently using myself as an example and to avoid calling any specific person out
I agree that I’d be shocked if GDM was training on eval sets. But I do think hill climbing on benchmarks is also very bad for those benchmarks being an accurate metric of progress and I don’t trust any AI lab not to hill climb on particularly flashy metrics
I’m not trying to agree with that one. I think that if someone has thought a bunch about the general topic of AI and has a bunch of useful takes. They can probably convert this on the fly to something somewhat useful, even if it’s not as reliable as it would be if they spent a long time thinking about it. Like I think I can give useful technical mechanistic interpretability takes even if the question is about topics I’ve not spent much time thinking about before
Are the joint names separated by spaces if not, the tokenization is going to be totally broken more generally I would be interested to see this Tried with a code that EG maps familiar tokens to obscure ones or something like mapping token with id k to id maximum minus K. Tokens feel like the natural way in llm would represent its processing and thus encoded processing. Doing things in individual letters is kind of hard
I had not noticed that part. Thanks for flagging
Thanks a lot for doing this. This is substantially more evaluation awareness than I would have predicted. I’m not super convinced by the classifying transcript purpose experiments, since the evaluator model is plausibly primed to think about this stuff, but the monitoring results seem compelling and very concerning. Thanks a lot for doing this work. I guess we really get concerned when it stops showing up in the chain of thought...
I would disagree that either one or four was achieved because, to my knowledge, the auditing game focused on finding, not fixing. It’s also a bit ambiguous whether the prediction involves fixing with interpretability or fixing with unrelated means. It wouldn’t surprise me if you could use the SAE-derived insights to filter out the relevant data if you really wanted to, but I’d guess an LLM classifier is more effective. Did they do that in the paper?
(To be clear, I think it’s a great paper, and to the degree that there’s a disagreement here it’s that I think your predictions weren’t covering the right comparative advantages of interp)
This is fantastic work, I’d be very excited to see more work in the vein of auditing games. It seems like the one of the best ways so far to test how useful different techniques for understanding models are
I found it helpful because it put me in the frame of a alien biological intelligence rather than an AI because I have lots of preconceptions about AIs and it’s it’s easy to implicitly think in terms of expected utility maximizers or tools or whatever. While if I’m imagining an octopus, I’m kind of imagining humans, but a bit weirder and more alien, and I would not trust humans
Oh sure, an executive assistant i.e. personal assistant in a work context can be super valuable just from an impact maximisation perspective but generally they need to be hired by your employer not by you in your personal capacity (unless you have a much more permissive/low security employer than Google)
Agreed. If I’m talking to someone who I expect to be able to recalibrate, I just explain that I think the standard norms are dumb, the norms I actually follow, and then give an honest and balanced assessment. If I’m talking to someone I don’t really know, I generally give a positive but not very detailed reference or don’t reply, depending on context.
Strong agree (except in that liking someone’s company is evidence that they would be a pleasant co-worker, but that’s generally not a high order bit). I find it very annoying that standard reference culture seems to often imply giving extremely positive references unless someone was truly awful, since it makes it much harder to get real info from references
Probably is but I can’t think of anything immediately
Idk, I personally feel near maxed out on spending money to increase my short term happiness (or at least, any ways coming to mind seem like a bunch of effort, like hiring a great personal assistant), and so the only reason to care about keeping it around is saving it for future use. I would totally be spending more money on myself now if I thought it would actually improve my life
In my incredibly biased opinion, the GDM AGI safety team is great and an effective place to work on reducing AI x-risk, and I would love to get applications from people here
I think it’s pretty plausible that something pathological like that is happening. We’re releasing this as an interesting idea that others might find useful for their use case, not as something we’re confident is a superior method. If we were continuing with SAE work, we would likely sanity check it more but we thought it better to release it than not