Good point. Maybe I should have said, we have to believe that there is at least a chance that it wasn’t a fluke.
If you are pretty sure it’s a fluke, then what is there to study, and why bother?
On the other hand, if you’re certain it’s a trend, there is still a lot to study: What caused the trend? How much control do we have over the causes? Etc.
I agree that, if progress was somehow a fluke, then progress studies would ideally find that out. We should follow reason wherever it leads, and be open to the truth whatever it is.
But I think the motivation for the field is the idea that it’s not a fluke, and empirically it seems to me that most people interested in the field see it as a trend.
Isn’t it both possible that it’s a fluke and also that going forward we can figure out mechanisms to promote it systematically?
To be clear, I think it’s more likely that not that a nontrivial fraction of recent progress has non fluke causes. I’m just also noting that the goal of enhancing progress seems at least partly disjoint from whether recent progress was a fluke.
Sorry I was unclear. I was actually imagining two possible scenarios.
The first would be deeper investigation reveals that recent progress mostly resulted from serendipity and lucky but more contingent than we expected historical factors. For example, maybe it turns out that the creation of industrial labs all hinged on some random quirk of the Delaware C-Corp code (I’m just making this up to be clear). Even though these factors were a fluke in the past and seem sort of arbitrary, we could still be systematic about bringing them about going forward.
The second scenario is even more pessimistic. Suppose we fail to find any factors that influenced recent progress—it’s just all noise. It’s hard to give an example of what this would look like because it would look like an absence of examples. Every rigorous investigation of a potential cause of historical progress would find a null result. Even in this pessimistic world, we still could say, “ok, well nothing in the past seemed to make a difference but we’re going to experiment to figure out things that do.”
That said, writing out this maximally pessimistic case made me realize how unlikely I think it is. It seems like we already know of certain factors which at least marginally increased the rate of progress, so I want to emphasize that I’m providing a line of retreat not arguing that this is how the world actually is.
Good point. Maybe I should have said, we have to believe that there is at least a chance that it wasn’t a fluke.
If you are pretty sure it’s a fluke, then what is there to study, and why bother?
On the other hand, if you’re certain it’s a trend, there is still a lot to study: What caused the trend? How much control do we have over the causes? Etc.
I agree that, if progress was somehow a fluke, then progress studies would ideally find that out. We should follow reason wherever it leads, and be open to the truth whatever it is.
But I think the motivation for the field is the idea that it’s not a fluke, and empirically it seems to me that most people interested in the field see it as a trend.
Isn’t it both possible that it’s a fluke and also that going forward we can figure out mechanisms to promote it systematically?
To be clear, I think it’s more likely that not that a nontrivial fraction of recent progress has non fluke causes. I’m just also noting that the goal of enhancing progress seems at least partly disjoint from whether recent progress was a fluke.
Maybe, what would that mean exactly? Or what’s an example of how that could be the case?
Sorry I was unclear. I was actually imagining two possible scenarios.
The first would be deeper investigation reveals that recent progress mostly resulted from serendipity and lucky but more contingent than we expected historical factors. For example, maybe it turns out that the creation of industrial labs all hinged on some random quirk of the Delaware C-Corp code (I’m just making this up to be clear). Even though these factors were a fluke in the past and seem sort of arbitrary, we could still be systematic about bringing them about going forward.
The second scenario is even more pessimistic. Suppose we fail to find any factors that influenced recent progress—it’s just all noise. It’s hard to give an example of what this would look like because it would look like an absence of examples. Every rigorous investigation of a potential cause of historical progress would find a null result. Even in this pessimistic world, we still could say, “ok, well nothing in the past seemed to make a difference but we’re going to experiment to figure out things that do.”
That said, writing out this maximally pessimistic case made me realize how unlikely I think it is. It seems like we already know of certain factors which at least marginally increased the rate of progress, so I want to emphasize that I’m providing a line of retreat not arguing that this is how the world actually is.