To clarify, were you saying here that we should have a strong prior that open source will progress quickly?
It reads like you’re saying that we should have a prior that it’s “good” which seems silly given that the reasons it was good before was that it lead to quicker progress, and the reasons it may start to be bad very soon is also that it will lead to quicker progress.
There was never any evidence one way or the other that open source code is better at avoiding being misapplied by state actors, terrorists, or generally irresponsible experimental projects, but it’s reasonable to guess for now that it’s not very good avoiding that.
It’s not obvious that open source leads to faster progress. Having high quality open source products reduces the incentives for private investment. I’m not sure in which regimes that will play out that it’s overall accelerationist, but I sort of guess that it will be decelerationist during an intense AI race (where the investments needed to push the frontier out are enormous and significantly profit-motivated).
Okay yeah, I meant quicker progress in expectation, I don’t believe that people today are capable of the level of coordination under which privatizing science could lead to faster progress in science.
But if we’re talking about mixed regimes, that’s a different question. Are we? Some do complain of a tilt towards a regime where frontier models will only be had by the private sphere, but it seems unlikely to happen.
To clarify, were you saying here that we should have a strong prior that open source will progress quickly?
It reads like you’re saying that we should have a prior that it’s “good” which seems silly given that the reasons it was good before was that it lead to quicker progress, and the reasons it may start to be bad very soon is also that it will lead to quicker progress.
There was never any evidence one way or the other that open source code is better at avoiding being misapplied by state actors, terrorists, or generally irresponsible experimental projects, but it’s reasonable to guess for now that it’s not very good avoiding that.
It’s not obvious that open source leads to faster progress. Having high quality open source products reduces the incentives for private investment. I’m not sure in which regimes that will play out that it’s overall accelerationist, but I sort of guess that it will be decelerationist during an intense AI race (where the investments needed to push the frontier out are enormous and significantly profit-motivated).
Okay yeah, I meant quicker progress in expectation, I don’t believe that people today are capable of the level of coordination under which privatizing science could lead to faster progress in science.
But if we’re talking about mixed regimes, that’s a different question. Are we? Some do complain of a tilt towards a regime where frontier models will only be had by the private sphere, but it seems unlikely to happen.