Scientific papers describe facts about the real world that aren’t fully determined by previous scientific papers.
Take for example the scientific papers describing a new species of bacteria that was unknown a decade earlier. Nothing in the training data describes it. You can also not determine the properties of the species based on first principles.
On the other hand, it might be possible to figure out an algorithm that does create texts that fit to given hash values.
If you are intelligent enough, you can deduce the laws of the universe from a surprisingly small amount of data. In the vein of your example, there is the story of Darwin deducing the existence of a moth with a long proboscis after seeing an orchid with a particular shape, and proving to be right. Perhaps papers from pre-2010 don’t have the right models, but maybe they have enough information and data for a sufficiently intelligent being to piece together from them whatever is missing?
Scientific papers describe facts about the real world that aren’t fully determined by previous scientific papers.
Take for example the scientific papers describing a new species of bacteria that was unknown a decade earlier. Nothing in the training data describes it. You can also not determine the properties of the species based on first principles.
On the other hand, it might be possible to figure out an algorithm that does create texts that fit to given hash values.
If you are intelligent enough, you can deduce the laws of the universe from a surprisingly small amount of data. In the vein of your example, there is the story of Darwin deducing the existence of a moth with a long proboscis after seeing an orchid with a particular shape, and proving to be right. Perhaps papers from pre-2010 don’t have the right models, but maybe they have enough information and data for a sufficiently intelligent being to piece together from them whatever is missing?
You can piece together some things, but there’s a lot of randomness in our world. A lot of important science is about discovering black swans.
Some things is enough, you’d still get less loss if you’re just right about the stuff that can be pieced together.