Idk, I personally feel near maxed out on spending money to increase my short term happiness (or at least, any ways coming to mind seem like a bunch of effort, like hiring a great personal assistant), and so the only reason to care about keeping it around is saving it for future use. I would totally be spending more money on myself now if I thought it would actually improve my life
Neel Nanda
In my incredibly biased opinion, the GDM AGI safety team is great and an effective place to work on reducing AI x-risk, and I would love to get applications from people here
On the other hand, if you have shorter timelines and higher P Doom, the value of saving for retirement becomes much lower, which means that if you earn a income notably higher than your needs, the cost of cryonics is much lower, If you don’t otherwise have valuable things to spend money on, they that get you value right now
I was also thinking recently that I would love this to exist! If I ever had the time I was going to try hacking it together in cursor
Huh, seems to be working for me. What do you see when you click on it?
I think it’s just not worth engaging with his claims about the limits of AI, he’s clearly already decided on his conclusion
Control space
For posterity, this turned out to be a very popular technique for jailbreaking open source LLMs—see this list of the 2000+ “abliterated” models on HuggingFace (abliteration is a mild variant of our technique someone coined shortly after, I think the main difference is that you do a bit of DPO after ablating the refusal direction to fix any issues introduced?). I don’t actually know why people prefer abliteration to just finetuning, but empirically people use it, which is good enough for me to call it beating baselines on some metric.
Interesting. Does anyone know what the counterparty risk is like here? Eg, am I gambling on the ETF continuing to be provided, the option market maker not going bust, the relevant exchange continuing to exist, etc. (the first and third generally seem like reasonable bets, but in a short timelines world everything is high variance...)
Yeah, if you’re doing this, you should definitely pre compute and save activations
I’ve been really enjoying voice to text + LLMs recently, via a great Mac App called Super Whisper (which can work with local speech to text models, so could also possibly be used for confidential stuff) - combining Super Whisper and Claude and Cursor means I can just vaguely ramble at my laptop about what experiments should happen and they happen, it’s magical!
I agree that OpenAI training on Frontier Math seems unlikely, and not in their interests. The thing I find concerning is that having high quality evals is very helpful for finding capabilities improvements—ML research is all about trying a bunch of stuff and seeing what works. As benchmarks saturate, you want new ones to give you more signal. If Epoch have a private benchmark they only apply to new releases, this is fine, but if OpenAI can run it whenever they want, this is plausibly fairly helpful for making better systems faster, since this makes hill climbing a bit easier.
This looks extremely comprehensive and useful, thanks a lot for writing it! Some of my favourite tips (like clipboard managers and rectangle) were included, which is always a good sign. And I strongly agree with “Cursor/LLM-assisted coding is basically mandatory”.
I passed this on to my mentees—not all of this transfers to mech interp, in particular the time between experiments is often much shorter (eg a few minutes, or even seconds) and often almost an entire project is in de-risking mode, but much of it transfers. And the ability to get shit done fast is super important
This seems fine to me (you can see some reasons I like Epoch here). My understanding is that most Epoch staff are concerned about AI Risk, though tend to longer timelines and maybe lower p(doom) than many in the community, and they aren’t exactly trying to keep this secret.
Your argument rests on an implicit premise that Epoch talking about “AI is risky” in their podcast is important, eg because it’d change the mind of some listeners. This seems fairly unlikely to me—it seems like a very inside baseball podcast, mostly listened to by people already aware of AI risk arguments, and likely that Epoch is somewhat part of the AI risk-concerned community. And, generally, I don’t think that all media produced by AI risk concerned people needs to mention that AI risk is a big deal—that just seems annoying and preachy. I see Epoch’s impact story as informing people of where AI is likely to go and what’s likely to happen, and this works fine even if they don’t explicitly discuss AI risk
I don’t know much about CTF specifically, but based on my maths exam/olympiad experience I predict that there’s a lot of tricks to go fast (common question archetypes, saved code snippets, etc) that will be top of mind for people actively practicing, but not for someone with a lot of domain expertise who doesn’t explicitly practice CTF. I also don’t know how important speed is for being a successful cyber professional. They might be able to get some of this speed up with a bit of practice, but I predict by default there’s a lot of room for improvement.
Tagging @philh @bilalchughtai @eventuallyalways @jbeshir in case this is relevant to you (though pooling money to get GWWC interested in helping may make more sense, if it can enable smaller donors and has lower fees)
For anyone considering large-ish donations (in the thousands), there are several ways to do this in general for US non-profits, as a UK taxpayer. (Several of these also work for people who pay tax in both the US and UK)
The one I’d recommend here is using the Anglo-American Charity—you can donate to them tax-deductibly (a UK charity) and they’ll send it to a US non-profit. I hear from a friend that they’re happy to forward it to every.org so this should be easy here. The main annoying thing is fees—for amounts below £15K it’s min(4%, £250) (so 4% above £6250, a flat fee of £250 below).
Fee structure; 4% on donations under £15k, 3% on gifts between £15,001 to £50,000 and 2% on gifts over £50k. Minimum gift of £1,000. Minimum fee £250.
The minimum donation is £1,000.00.
But this is still a major saving, even in the low thousands, especially if you’re in a high tax bracket. Though it may make more sense for you to donate to another charity.
Another option is the charitable aid foundation’s donor advised gift, which is a similar deal with worse fees: min(4%, £400) on amounts below £150K.
If you’re a larger donor (eg £20K+) it may make sense to set up a donor advised fund, which will often let you donate to worldwide non-profits.
Sure, but I think that human cognition tends to operate at a level of abstract above the configuration of atoms in a 3D environment. Like “that is a chair” is a useful way to reason about an environment. Whilethat “that is a configuration of pixels that corresponds to a chair when projected at a certain angle in certain lighting conditions” must first be converted to “that is a chair” before anything useful can be done. Text just has a lot of useful preprocessing applied already and is far more compressed
Strong +1, that argument didn’t make sense to me. Images are a fucking mess—they’re a grid of RGB pixels, of a 3D environment (interpreted through the lens of a camera) from a specific angle. Text is so clean and pretty in comparison, and has much richer meaning, and has a much more natural mapping to concepts we understand
Probably is but I can’t think of anything immediately