I suspect you use the term “insight” to describe something that I would classify as a hypothesis rather than observation (evidence is a particular kind of observation, yes?).
Consider Pythagoras’ theorem and an agent without any knowledge of it. If you provide the agent with the length of the legs of a right-angled triangle and ask for the length of the hypotenuse, it will use some other algorithm/heuristic to reach an answer (probably draw and measure a similar triangle).
Now you suggest the theorem to the agent. This suggestion is in itself evidence for the theorem, if for no other reason then because P(hypothesis H | H is mentioned) > P(H | H is not mentioned). Once H steals some of the probability from competing hypotheses, the agent looks for more evidence and updates it’s map.
Was his first answer “rational”? I believe it was rational enough. I also think it is a type error to compare hypotheses and evidence.
If you define “rational” as applying the best heuristic you have, you still need a heuristic for choosing a heuristic to use (i.e. read wikipedia, ask expert, become expert, and so on). If you define it as achieving maximum utility, well, then it’s pretty subjective (but can still be measured). I’d go for the latter.
P.S. Can Occam’s razor (or any formal presentation of it) be classified as a hypothesis? Evidence for such could be any observation of a simpler hypotheses turning out to be a better one, similar for evidence against. If that is true, then you needn’t dual-wield the sword of Bayes and Occam’s razor; all you need is one big Bayesian blade.
P.S. Can Occam’s razor (or any formal presentation of it) be classified as a hypothesis?
Sadly, no; this is the “problem of induction” and to put it briefly, if you try to do what you suggest you end up having to assume what you’re trying to prove. If you start with a “flat” prior in which you consider every possible Universe-history to be equally likely, you can’t collect evidence for Occam’s razor. The razor has to be built in to your priors. Thus, Solomonoff’s lightsaber.
Sweet, but according to the wiki the lightsaber doesn’t include full Bayesian reasoning, only the special case where the likelihood ratio of evidence is zero.
One could argue that you can reach the lightsaber using the Bayesian blade, but not vice versa.
I suspect you use the term “insight” to describe something that I would classify as a hypothesis rather than observation (evidence is a particular kind of observation, yes?).
Consider Pythagoras’ theorem and an agent without any knowledge of it. If you provide the agent with the length of the legs of a right-angled triangle and ask for the length of the hypotenuse, it will use some other algorithm/heuristic to reach an answer (probably draw and measure a similar triangle).
Now you suggest the theorem to the agent. This suggestion is in itself evidence for the theorem, if for no other reason then because P(hypothesis H | H is mentioned) > P(H | H is not mentioned). Once H steals some of the probability from competing hypotheses, the agent looks for more evidence and updates it’s map.
Was his first answer “rational”? I believe it was rational enough. I also think it is a type error to compare hypotheses and evidence.
If you define “rational” as applying the best heuristic you have, you still need a heuristic for choosing a heuristic to use (i.e. read wikipedia, ask expert, become expert, and so on). If you define it as achieving maximum utility, well, then it’s pretty subjective (but can still be measured). I’d go for the latter.
P.S. Can Occam’s razor (or any formal presentation of it) be classified as a hypothesis? Evidence for such could be any observation of a simpler hypotheses turning out to be a better one, similar for evidence against. If that is true, then you needn’t dual-wield the sword of Bayes and Occam’s razor; all you need is one big Bayesian blade.
Sadly, no; this is the “problem of induction” and to put it briefly, if you try to do what you suggest you end up having to assume what you’re trying to prove. If you start with a “flat” prior in which you consider every possible Universe-history to be equally likely, you can’t collect evidence for Occam’s razor. The razor has to be built in to your priors. Thus, Solomonoff’s lightsaber.
“Solomonoff’s lightsaber”
Sweet, but according to the wiki the lightsaber doesn’t include full Bayesian reasoning, only the special case where the likelihood ratio of evidence is zero.
One could argue that you can reach the lightsaber using the Bayesian blade, but not vice versa.
The lightsaber does include full Bayesian reasoning.