I believe AI pauses by governments would absolutely be more serious and longer, preventing overhangs from building up too much.
Are you saying that overhangs wouldn’t build up too much under pauses because the government wouldn’t let it happen, or that RSPs would have less overhang because they’d pause for less long so less overhang would build up? I can’t quite tell.
I’m not saying there’s no reason to think that RSPs are better or worse than pause, just that if overhang is a relevant consideration for pause, it’s also a relevant consideration for RSPs.
I’d imagine that RSP proponents think that if we execute them properly, we will simply not build dangerous models beyond our control, period.
I think pause proponents think similarly!
Do you see the same people invoking overhang as an argument against pauses and also talking about RSPs as though they are not also impacted?
I guess I’m not tracking this closely enough. I’m not really that focussed on any one arguer’s individual priorities, but more about the discourse in general. Basically, I think that overhang is a consideration for unconditional pauses if and only if it’s a consideration for RSPs, so it’s a bad thing if overhang is brought up as an argument against unconditional pauses and not against RSPs, because this will distort the world’s ability to figure out the costs and benefits of each kind of policy.
Also, to be clear, it’s not impossible that RSPs are all things considered better than unconditional pauses, and better than nothing, despite overhang. But if so, I’d hope someone somewhere would have written a piece saying “RSPs have the cost of causing overhang, but on net are worth it”.
I’m not saying that RSPs are or aren’t better than a pause. But I would think that if overhang is a relevant consideration for pauses, it’s also a relevant consideration for RSPs.
A complaint about AI pause: if we pause AI and then unpause, progress will then be really quick, because there’s a backlog of improvements in compute and algorithmic efficiency that can be immediately applied.
One definition of what an RSP is: if a lab makes observation O, then they pause scaling until they implement protection P.
Doesn’t this sort of RSP have the same problem with fast progress after pausing? Why have I never heard anyone make this complaint about RSPs? Possibilities:
They do and I just haven’t seen it
People expect “AI pause” to produce longer / more serious pauses than RSPs (but this seems incidental to the core structure of RSPs)
I continue to think that agent foundations research is kind of underrated. Like, we’re supposed to do mechinterp to understand the algorithm models implement—but how do we know what algorithms are good?
Ah, thanks for clarifying that.
I’m kind of confused which unilateralist got to design the game. You say:
The first person to click the Unilateral Virtue Link was a proponent of “Avoiding actions that noticeably increase the chance that the world will end.” But, this virtue was actually in the majority. The first unilateralist of a Virtue Minority was a proponent of “Accurately reporting your epistemic state.”
A year later, as we decided what to do for Petrov Day, we decided to lure the first unilateralist into a surprise meeting, where I then said “Here’s a reminder of what happened in Petrov Day last year. You now have one hour to design this year’s Petrov game. Go.”
So it sounds like the unilateralist who wanted to avoid actions that noticeably increase the chance the world will end got picked. But then it sounds like the winner made a game that was supposed to be about accurately reporting epistemic state:
The designer had (I think?) initially noticed the “focus on accurately reporting epistemic state” aspect, but said that during the stressful hour of designing the game had eventually forgotten that. The version they handed off wasn’t particularly optimized for that, but the framework of a social deception game seemed to me to be a good substrate for “accurate epistemic reporting.” [...] It seemed important to me that Petrov’s payoff specifically be about reporting his beliefs
I’d now make this bet if you were down. Offer expires in 48 hours.
I’d like to wait and see what various models say.
[this comment is irrelevant to the point you actually care about and is just nit-picking about the analogy]
There is a pretty big divide between “liberal” and “conservative” Christianity that is in some ways bigger than the divide between different denominations. In the US, people who think of themselves as “Episcopalians” tend to be more liberal than people who call themselves “Baptists”. In the rest of this comment, I’m going to assume we’re talking about conservative Anglicans rather than Episcopalians (those terms referring to the same denominational family), and also about conservative Baptists, since they’re more likely to be up to stuff / doing meaningful advocacy, and more likely to care about denominational distinctions. That said, liberal Episcopalians and liberal Baptists are much more likely to get along, and also openly talk about how they’re in cooperation.
My guess is that conservative Anglicans and Baptists don’t spend much time at each other’s churches, at least during worship, given that they tend to have very different types of services and very different views about the point of worship (specifically about the role of the eucharist). Also there’s a decent chance they don’t allow each other to commune at their church (more likely on the Baptist end). Similarly, I don’t think they are going to have that much social overlap, altho I could be wrong here. There’s a good chance they read many of the same blogs tho.
In terms of policy advocacy, on the current margin they are going to mostly agree—common goals are going to be stuff like banning abortion, banning gay marriage, and ending the practice of gender transition. Anglican groups are going to be more comfortable with forms of state Christianity than Baptists are, altho this is lower-priority for both, I think. They are going to advocate for their preferred policies in part by denominational policy bodies, but also by joining common-cause advocacy organizations.
Both Anglican and Baptist churches are largely going to be funded by members, and their members are going to be disjoint. That said it’s possible that their policy bodies will share large donor bases.
They are also organized pretty differently internally: Anglicans have a very hierarchical structure, and Baptists having a very decentralized structure (each congregation is its own democratic policy, and is able to e.g. vote to remove the pastor and hire a new one)
Anyway: I’m pretty sympathetic to the claim of conservative Anglicans and Baptists being meaningfully distinct power bases, altho it would be misleading to not acknowledge that they’re both part of a broader conservative Christian ecosystem with shared media sources, fashions, etc.
Part of the reason this analogy didn’t vibe for me is that Anglicans and Baptists are about as dissimilar as Protestants can get. If it were Anglicans and Presbyterians or Baptists and Pentecostals that would make more sense, as those denominations are much more similar to each other.
Further updates:
On the one hand, Nate Silver’s model now gives Trump a ~30% chance of winning in Virginia, making my side of the bet look good again.
On the other hand, the Economist model gives Trump a 10% chance of winning Delaware and a 20% chance of winning Illinois, which suggests that there’s something going wrong with the model and that it was untrustworthy a month ago.
That said, betting markets currently think there’s only a one in four chance that Biden is the nominee, so this bet probably won’t resolve.
I will spend time reading posts and papers, improving coding skills as needed to run and interpret experiments, learning math as needed for writing up proofs, talking with concept-based interpretability researchers as well as other conceptual alignment researchers
I feel like this is missing the bit where you write proofs, run and interpret experiments, etc.
As a maximal goal, I might seek to test my theories about the detection of generalizable human values (like reciprocity and benevolence) by programming an alife simulation meant to test a toy-model version of agentic interaction and world-model agreement/interoperability through the fine-structure of the simulated agents.
Do you think you will be able to do this in the next 6 weeks? Might be worth scaling this down to “start a framework to test my theories” or something like that
Realized that I didn’t respond to this—PauseAI’s proposal is for a pause until safety can be guaranteed, rather than just for 6 months.