Running Lightcone Infrastructure, which runs LessWrong. You can reach me at habryka@lesswrong.com. I have signed no contracts or agreements whose existence I cannot mention.
habryka
Karma: 38,063
I think “full visibility” seems like the obvious thing to ask for, and something that could maybe improve things. Also, preventing you from selling your products to the public, and basically forcing you to sell your most powerful models only to the government, gives the government more ability to stop things when it comes to it.
I will think more about this, I don’t have any immediate great ideas.
If the project was fueled by a desire to beat China, the structure of the Manhattan project seems unlikely to resemble the parts of the structure of the Manhattan project that seemed maybe advantageous here, like having a single government-controlled centralized R&D effort.
My guess is if something like this actually happens, it would involve a large number of industry subsidies, and would create strong institutional momentum that even when things got dangerous, to push the state of the art forward, and in as much as there is pushback, continue dangerous development in secret.
In the case of nuclear weapons the U.S. really went very far under the advisement of Edward Teller, so I think the outside view here really doesn’t look good:
I don’t remember ever adjudicating this, but my current intuition, having not thought about it hard, is that I don’t see a super clear line here (like, in a moderation dispute I can imagine judging either way depending on the details).
The Truman Show: Great depiction of crisis of faith, noticing your confusion, and generally is about figuring out the truth.
Most relevant sequence posts: Crisis of Faith, Lonely Dissent
Going by today’s standards, we should have banned Gwern in 2012.
(I don’t understand what this is referring to)
Indeed. I fixed it. Let’s see whether it repeats itself (we got kind of malformed HTML from the RSS feed).
Update: I have now cross-referenced every single email for accuracy, cleaned up and clarified the thread structure, and added subject lines and date stamps wherever they were available. I now feel comfortable with people quoting anything in here without checking the original source (unless you are trying to understand the exact thread structure of who was CC’d and when, which was a bit harder to compress into a linear format).
(For anyone curious, the AI transcription and compilation made one single error, which is that it fixed a typo in one of Sam’s messages from “We did this is a way” to “We did this in a way”. Honestly, my guess is any non-AI effort would have had a substantially higher error rate, which was a small update for me on the reliability of AI for something like this, and also makes the handwringing about whether it is OK post something like this feel kind of dumb. It also accidentally omitted one email with a weird thread structure.)
FWIW, my best guess is the document contains fewer errors than having a human copy-paste things and stitch it together. The errors have a different nature to them, and so it makes sense to flag them, but like, I started out with copy-pasting and OCR, and that did not actually have an overall lower error rate.
If other people have to check it before they quote it, why is it OK for you not to check it before you post it?
Because I said prominently at the top that I used AI assistance for it. Of course, feel free to do the same.
Fixed! That specific response had a very weird thread structure, so makes sense the AI I used got confused. Plausible something else is still missing, though I think I’ve now read through all the original PDFs and didn’t see anything new.
OpenAI Email Archives (from Musk v. Altman)
What do you mean by “applied research org”? Like, applied alignment research?
Using Dangerous AI, But Safely?
A bunch of very interesting emails between Elon, Sam Altman, Ilya and Greg were released (I think in some legal proceedings, but not sure). It would IMO be cool for someone to gather them all and do some basic analysis of them.
This was a really good analysis of a bunch of election stuff that I hadn’t seen presented clearly like this anywhere else. If it wasn’t about elections and news I would curate it.
Not sure what you mean. The API continues to exist (and has existed since the beginning of LW 2.0).
I think the comment more confirms than disconfirms John’s comment (though I still think it’s too broad for other reasons). OP “funding” something historically has basically always meant recommending a grant to GV. Luke’s language to me suggests that indeed the right of center grants are no longer referred to GV (based on a vague vibe of how he refers to funders in plural).
OP has always made some grant recommendations to other funders (historically OP would probably describe those grants as “rejected but referred to an external funder”). As Luke says, those are usually ignored, and OP’s counterfactual effect on those grants is much less, and IMO it would be inaccurate to describe those recommendations as “OP funding something”. As I said in the comment I quote in the thread, most OP staff would like to fund things right of center, but GV does not seem to want to, as such the only choice OP has is to refer them to other funders (which sometimes works, but mostly doesn’t).
As another piece of evidence, when OP defunded all the orgs that GV didn’t want to fund anymore, the communication emails that OP sent said that “Open Philanthropy is exiting funding area X” or “exiting organization X”. By the same use of language, yes, it seems like OP has exited funding right-of-center policy work.
(I think it would make sense to taboo “OP funding X” in future conversations to avoid confusion, but also, I think historically it was very meaningfully the case that getting funded by GV is much better described as “getting funded by OP” given that you would never talk to anyone at GV and the opinions of anyone at GV would basically have no influence on you getting funded. Things are different now, and in a meaningful sense OP isn’t funding anyone anymore, they are just recommending grants to others, and it matters more what those others think then what OP staff thinks)
One of these types of orgs is developing a technology with the potential to kill literally all of humanity. The other type of org is funding research that if it goes badly mostly just wasted their own money. Of course the demands for legibility and transparency should be different.
My best guess this is false. As a quick sanity-check, here are some bipartisan and right-leaning organizations historically funded by OP:
FAI leans right. https://www.openphilanthropy.org/grants/foundation-for-american-innovation-ai-safety-policy-advocacy/
Horizon is bipartisan https://www.openphilanthropy.org/grants/open-philanthropy-technology-policy-fellowship-2022/ .
CSET is bipartisan https://www.openphilanthropy.org/grants/georgetown-university-center-for-security-and-emerging-technology/ .
IAPS is bipartisan. https://www.openphilanthropy.org/grants/page/2/?focus-area=potential-risks-advanced-ai&view-list=false, https://www.openphilanthropy.org/grants/institute-for-ai-policy-strategy-general-support/
RAND is bipartisan. https://www.openphilanthropy.org/grants/rand-corporation-emerging-technology-fellowships-and-research-2024/.
Safe AI Forum. https://www.openphilanthropy.org/grants/safe-ai-forum-operating-expenses/
AI Safety Communications Centre. https://www.openphilanthropy.org/grants/effective-ventures-foundation-ai-safety-communications-centre/ seems to lean left.
Of those, I think FAI is the only one at risk of OP being unable to fund them, based on my guess of where things are leaning. I would be quite surprised if they defunded the other ones on bipartisan grounds.
Possibly you meant to say something more narrow like “even if you are trying to be bipartisan, if you lean right, then OP is substantially less likely to fund you” which I do think is likely true, though my guess is you meant the stronger statement, which I think is false.
Yeah, IMO we should just add a bunch of functionality for integrating alignment forum stuff more with academic things. It’s been on my to do list for a long time.