Investing in early-stage AGI companies helps with reducing x-risk (via mission hedging, having board seats, shareholder activism)
Jonas V
Karma: 1,327
Yeah, that does also feel right to me. I have been thinking about setting up some fund that maybe buys up a bunch of the equity that’s held by safety researchers, so that the safety researchers don’t have to also blow up their financial portfolio when they press the stop button or do some whistleblowing or whatever, and that does seem pretty incentive wise.
I’m interested in helping with making this happen.
Very interesting conversation!
I’m surprised by the strong emphasis of shorting long-dated bonds. Surely there’s a big risk of nominal interest rates coming apart from real interest rates, i.e. lots of money getting printed? I feel like it’s going to be very hard to predict what the Fed will do in light of 50% real interest rates, and Fed interventions could plausibly hurt your profits a lot here.
(You might suggest shorting long-dated TIPS, but those markets have less volume and higher borrow fees.)
Some additional people reached out to me—just reiterating that I’m happy to do more at 20:1 odds!
Happy to do another $40k at 55:1 odds if you like (another $727), and another $20k at 20:1 odds after that.
Confirm.
Happy to bet $40k at
110:120:1 odds ($364$2k). (Edited Sep 2023; previous bets confirmed at previous odds.)USDC ERC-20 (Ethereum): (address removed for privacy, please DM if you want to trade more)
USDC Polygon: (address removed for privacy, please DM if you want to trade more)
(Edit 23 June 3:45 PT): I’m only willing to bet assuming that AGI-created tech doesn’t count for the purposes of this bet—it has to be something more supernatural than that.)
I think SBF rarely ever fired anyone, so “kicked out” seems wrong, but I heard that people who weren’t behaving in the way SBF liked (e.g., recklessly risk-taking) got sidelined and often left on their own because their jobs became unpleasant or they had ethical qualms, which would be consistent with evaporative cooling.
High schoolers can apply to the Atlas Fellowship: $10k scholarship + 11-day program
Yeah, I fully agree with this, and am aware of the cost. Apologies once more for not jumping in sooner when I wasn’t sure whether applicants had been emailed by my colleague or not.
This workshop will be postponed to January, likely to 6–9 Jan. GradientDissenter was planning to give an update to all applicants; I hope they will do so soon. I understand that some of you may have made their plans hoping to be able to participate in the workshop or were otherwise hoping for a fast response, and I apologize for completely missing the deadline, the lack of communication, and the change of plans.
(Why did this happen? Evaluating applications ended up being harder than anticipated, and I failed to jump in and fix things when the workshop planning wasn’t progressing as planned, partly because I was on vacation.)
We might if it goes well. If you want to be pinged if we run one, please submit a quick application through our form!
Thanks for the feedback! I’ve edited the post to clarify where the funding is coming from and who is running this.
Regarding the content, one of my co-organizers may leave another comment later. The short version is that we’ll be re-running some of the most popular content from previous workshops, but primarily focus on informal conversations, as participants usually rate that as much more useful than the actual content of the workshop.
It’s funded by the Atlas Fellowship, which is funded by Open Philanthropy. It’s something of a side-hustle of Atlas (outside of the scope and brand of the organization, and run by a subset of our team and some external collaborators). We have a fair amount of experience running different kinds of workshops, and are experimenting with what programs targeted at other demographic and niches might look like.
Thanks for the feedback! Added to the OP.
“Career networking” feels like it encompasses some useful stuff, like hearing about new opportunities, meeting potential co-founders, etc.
It also sounds like it encompasses some bad stuff, like a race to get the most connections and impressing people.
We’re going to try and have some of the useful kind of career networking, and to push hard against the strong pressures towards the bad kind of career networking.
There also aren’t that many careers out there just waiting for an employee to come slot into a well-defined role that actually makes progress on preventing x-risk or similar, so we’re much more excited about helping people carve out their own path, not in connecting them to to employers running hiring rounds.
Is there going to be a digital option?
Unfortunately we’re not going to be able to accommodate that. It’s fully in-person, since a large part of the point is in-person interactions.
Yes. (Within reason.)
LessWrong readers are invited to apply to the Lurkshop
Would be very excited to get applications to the EA Infrastructure Fund (EAIF)! Apply here, it’s fast and easy.
(I run EA Funds, which includes EAIF.)
Having another $1 billion to prevent AGI x-risk would be useful because we could spend it on large compute budgets for safety research teams.