But that’s going to bite you every time you don’t have the luxury of being able to apply these hacks—say because you’re modelling (some aspect of) human history, and can’t rerun the experiment.
? History sounds like exactly the situation where “hold back half the data, hypothsise on the other half, then look at the whole” is the only way of reasonably going about this.
Also, you won’t be able to build an AGI.
Don’t follow that argument at all—in the worst case scenario, you can brute force it by scanning and moddelling a human brain. But even if true, it’s not really an issue for social scientists and their ilk. And there the “look at half the data” would cause definite improvements in their proceedures. It would make science work for the “flawed but honest” crowd.
As for deliberately holding back half the data from other scientists (as opposed to one guy simply choosing to only look at half), that’s a different issue. I’ve got no really strong feelings on that. It could go either way.
It’s an ok hack for someone in the “flawed but honest” crowd, individually. But note that it really doesn’t scale to allowing you to deal with corruption (which was one of the problems I assumed in the post you replied to).
Extended to an entire field, this means that you may end up with N papers, all about the same data set, all proposing a different hypothesis that produces a good match on the set, and all of them claiming that their hypothesis was formulated using this procedure. IOW, you end up with unverifiable “trust us, we didn’t cheat” claims for each of those hypotheses. Which is not a good basis for arriving at a consensus in the field.
Re AI design, assuming you actually understand what you implemented (as opposed to just blindly copying algorithms from the human brain without understanding what they do), the reason this method would work is that you’ve successfully extracted the human built-in simplicity prior (and I don’t know how good that one is exactly, but it has to be a halfway workable approximation; otherwise humans couldn’t model reality at all).
? History sounds like exactly the situation where “hold back half the data, hypothsise on the other half, then look at the whole” is the only way of reasonably going about this.
Don’t follow that argument at all—in the worst case scenario, you can brute force it by scanning and moddelling a human brain. But even if true, it’s not really an issue for social scientists and their ilk. And there the “look at half the data” would cause definite improvements in their proceedures. It would make science work for the “flawed but honest” crowd.
As for deliberately holding back half the data from other scientists (as opposed to one guy simply choosing to only look at half), that’s a different issue. I’ve got no really strong feelings on that. It could go either way.
It’s an ok hack for someone in the “flawed but honest” crowd, individually. But note that it really doesn’t scale to allowing you to deal with corruption (which was one of the problems I assumed in the post you replied to).
Extended to an entire field, this means that you may end up with N papers, all about the same data set, all proposing a different hypothesis that produces a good match on the set, and all of them claiming that their hypothesis was formulated using this procedure. IOW, you end up with unverifiable “trust us, we didn’t cheat” claims for each of those hypotheses. Which is not a good basis for arriving at a consensus in the field.
Re AI design, assuming you actually understand what you implemented (as opposed to just blindly copying algorithms from the human brain without understanding what they do), the reason this method would work is that you’ve successfully extracted the human built-in simplicity prior (and I don’t know how good that one is exactly, but it has to be a halfway workable approximation; otherwise humans couldn’t model reality at all).