If by “account for that” you mean not be in direct conflict with earlier sense data, then sure. All tautologies about the data will continue to be true. Suppose some data can be predicted by classical mechanics with 75% accuracy. This is a tautology given the data itself, and no future theory will somehow make classical mechanics stop giving 75% accurate predictions for that past data.
Maybe that’s all you meant?
I’d sort of interpreted you as asking questions about properties of the theory. E.g. “this data is really well explained by the classical mechanics of point particles, therefore any future theory should have a particularly simple relationship to the point particle ontology.” It seems like there shouldn’t be a guaranteed relationship that’s much simpler than reconstructing the data and recomputing the inferred point particles.
I spent a little while trying to phrase this in terms of Turing machines but I don’t think I quite managed to capture the spirit.
It seems like there shouldn’t be a guaranteed relationship that’s much simpler than reconstructing the data and recomputing the inferred point particles.
Yeah, I’m claiming exactly the opposite of this. When the old theory itself has some simple structure (e.g. classical mechanics), there should be a guaranteed relationship that’s much simpler than reconstructing the data and recomputing the inferred point particles.
One possible formulation: if I find that a terabyte of data compresses down to a gigabyte, and then I find a different model which compresses it down to 500MB, there should be a relationship between the two models which can be expressed without expanding out the whole terabyte. (Or, if there isn’t such a relationship, that means the two models are capturing different patterns from the data, and there should exist another model which compresses the data more than either by capturing the patterns found by both models.)
Right, it’s a little tricky to specify exactly what this “relationship” is. Is the notion that you should be able to compress the approximate model, given an oracle for the code of the best one (i.e. that they share pieces?). Because most Turing machines don’t compress well, and so it’s easy to find counterexamples (the most straightforward class is where the approximate model is already extremely simple).
Anyhow, like I said, hard to capture the spirit of the problem. But when I *do* try to formalize the problem, it tends to not have the property, which is definitely driving my intuition.
I’d expect Turing machines to be a bad way to model this. They’re inherently blackboxy; the only “structure” they make easy to work with is function composition. The sort of structures relevant here don’t seem like they’d care much about function boundaries. (This is why I use models like these as my default model of computation these days.)
Anyway, yeah, I’m still not sure what the “relationship” should be, and it’s hard to formulate in a way that seems to capture the core idea.
If by “account for that” you mean not be in direct conflict with earlier sense data, then sure. All tautologies about the data will continue to be true. Suppose some data can be predicted by classical mechanics with 75% accuracy. This is a tautology given the data itself, and no future theory will somehow make classical mechanics stop giving 75% accurate predictions for that past data.
Maybe that’s all you meant?
I’d sort of interpreted you as asking questions about properties of the theory. E.g. “this data is really well explained by the classical mechanics of point particles, therefore any future theory should have a particularly simple relationship to the point particle ontology.” It seems like there shouldn’t be a guaranteed relationship that’s much simpler than reconstructing the data and recomputing the inferred point particles.
I spent a little while trying to phrase this in terms of Turing machines but I don’t think I quite managed to capture the spirit.
Yeah, I’m claiming exactly the opposite of this. When the old theory itself has some simple structure (e.g. classical mechanics), there should be a guaranteed relationship that’s much simpler than reconstructing the data and recomputing the inferred point particles.
One possible formulation: if I find that a terabyte of data compresses down to a gigabyte, and then I find a different model which compresses it down to 500MB, there should be a relationship between the two models which can be expressed without expanding out the whole terabyte. (Or, if there isn’t such a relationship, that means the two models are capturing different patterns from the data, and there should exist another model which compresses the data more than either by capturing the patterns found by both models.)
Right, it’s a little tricky to specify exactly what this “relationship” is. Is the notion that you should be able to compress the approximate model, given an oracle for the code of the best one (i.e. that they share pieces?). Because most Turing machines don’t compress well, and so it’s easy to find counterexamples (the most straightforward class is where the approximate model is already extremely simple).
Anyhow, like I said, hard to capture the spirit of the problem. But when I *do* try to formalize the problem, it tends to not have the property, which is definitely driving my intuition.
I’d expect Turing machines to be a bad way to model this. They’re inherently blackboxy; the only “structure” they make easy to work with is function composition. The sort of structures relevant here don’t seem like they’d care much about function boundaries. (This is why I use models like these as my default model of computation these days.)
Anyway, yeah, I’m still not sure what the “relationship” should be, and it’s hard to formulate in a way that seems to capture the core idea.