Certainly some incremental research is very useful. But much of it isn’t. I’m not familiar with the ACL and EMNLP conferences, but for ML and statistics, there are large numbers of papers that don’t really contribute much (and these aren’t failed attempts at breakthroughs). You can see that this must be true from the sheer volume of papers now—there can’t possibly be that many actual advances.
For LDPC codes, it certainly was true that for years people didn’t realize their potential. But there wasn’t any good reason not to investigate—it’s sort of like nobody pointing a telescope at Saturn because Venus turned out to be rather featureless, and why would Saturn be different? There was a bit of tunnel vision, with an unjustified belief that one couldn’t really expect much more than what the codes being investigated delivered—though one could of course publish lots of papers on a new variation in sequential decoding of convolutional codes. (There was good evidence that this would never lead to the Shannon limit—but that of course must surely be unobtainable...)
Regarding MCMC and Bayesian inference, I think there was just nobody making the connection—nobody who actually knew what the methods from physics could do, and also knew what the computational obstacles for Bayesian inference were. I don’t think anyone thought of applying the Metropolis algorithm to Bayesian inference and then said, “but surely that wouldn’t work...”. It’s obviously worth a try.
Certainly some incremental research is very useful. But much of it isn’t. I’m not familiar with the ACL and EMNLP conferences, but for ML and statistics, there are large numbers of papers that don’t really contribute much (and these aren’t failed attempts at breakthroughs). You can see that this must be true from the sheer volume of papers now—there can’t possibly be that many actual advances.
For LDPC codes, it certainly was true that for years people didn’t realize their potential. But there wasn’t any good reason not to investigate—it’s sort of like nobody pointing a telescope at Saturn because Venus turned out to be rather featureless, and why would Saturn be different? There was a bit of tunnel vision, with an unjustified belief that one couldn’t really expect much more than what the codes being investigated delivered—though one could of course publish lots of papers on a new variation in sequential decoding of convolutional codes. (There was good evidence that this would never lead to the Shannon limit—but that of course must surely be unobtainable...)
Regarding MCMC and Bayesian inference, I think there was just nobody making the connection—nobody who actually knew what the methods from physics could do, and also knew what the computational obstacles for Bayesian inference were. I don’t think anyone thought of applying the Metropolis algorithm to Bayesian inference and then said, “but surely that wouldn’t work...”. It’s obviously worth a try.