Counterpoint—let’s say you have a proposal that is safer than the frontier in some respects, but doesn’t always generalize / scale. I imagine it would be better to submit the proposal anyways, while highlighting failure modes (scaling included). Submitting imperfect proposals in this way allows the community to poke holes, fix problematic aspects, and perhaps even discover new angles of attack while discussing the shortcomings.
The purpose of this post is to communicate, not to persuade. It may be that we want to bit [sic] the bullet of the strongest form of robustness to scale, and build an AGI that is simply not robust to scale, but if we do, we should at least realize that we are doing that.
While this may not be what is happening here, in general I think that when an author opens with “I acknowledge the following criticisms to my argument” they make it unfairly socially unacceptable to respond with “Yeah, but the criticisms.” I think this is a bad discourse norm, and people should give that response more often.
I was imagining someone with a bright yet flawed idea reading this post, realizing their idea doesn’t scale, and ending up scrapping something redeemable that people with more expertise could have steelmanned. I’m not presuming that Scott was advocating a “totally scalable or STFU” criterion, but I wanted to put that consideration out there.
Counterpoint—let’s say you have a proposal that is safer than the frontier in some respects, but doesn’t always generalize / scale. I imagine it would be better to submit the proposal anyways, while highlighting failure modes (scaling included). Submitting imperfect proposals in this way allows the community to poke holes, fix problematic aspects, and perhaps even discover new angles of attack while discussing the shortcomings.
While this may not be what is happening here, in general I think that when an author opens with “I acknowledge the following criticisms to my argument” they make it unfairly socially unacceptable to respond with “Yeah, but the criticisms.” I think this is a bad discourse norm, and people should give that response more often.
I was imagining someone with a bright yet flawed idea reading this post, realizing their idea doesn’t scale, and ending up scrapping something redeemable that people with more expertise could have steelmanned. I’m not presuming that Scott was advocating a “totally scalable or STFU” criterion, but I wanted to put that consideration out there.