I believe this sort of reasoning may be justified through a theory for probabilistic reasoning under restricted computational resources. Unfortunately I don’t think this theory has been developed yet, and it will probably be very difficult to do so. But I think there is some progress in this direction, for example there are lots of conjectures in number theory which are justified by probabilistic reasoning, see this excellent post by Terry Tao for some good examples.
I think that a suitable adjustment of Benford’s law is the right prior for a generic subset of the natural numbers, which leads one to conjecture the prime number theorem before even knowing the definition of a prime number (and also is why Benford’s law is so successful)! Once you do know the definition, and are willing to do a small amount of computation, you can do a sort of update with this new information to get better heuristics, just like in Terry Tao’s post linked above. The restriction on the amount of computation is necessary to make this idea useful; if you had unlimited computational resources, you would know everything about the primes already.
It’s exciting to see in number theory lots of work on finding good priors for these sorts of things, another good example is with the Cohen-Lenstra heuristics which are some conjectures ultimately based on the prior probability of a generic group of some kind having a certain property (sorry, I don’t remember very well). It seems like work dealing with the limited computational resources mostly has yet to be done, but there might be lots of good ideas in computational complexity theory (not my domain of expertise). I think that having this kind of theory will also be important to FAI research, since it will need to reason correctly about math with limited resources.
Anyway, I think you are right that Euler had enough evidence to be confident of his solution to the Basel problem. I hope that a solid theory for this sort of reasoning is found soon, and I’m glad to see that other people are thinking about it.
I believe this sort of reasoning may be justified through a theory for probabilistic reasoning under restricted computational resources. Unfortunately I don’t think this theory has been developed yet, and it will probably be very difficult to do so. But I think there is some progress in this direction, for example there are lots of conjectures in number theory which are justified by probabilistic reasoning, see this excellent post by Terry Tao for some good examples.
I think that a suitable adjustment of Benford’s law is the right prior for a generic subset of the natural numbers, which leads one to conjecture the prime number theorem before even knowing the definition of a prime number (and also is why Benford’s law is so successful)! Once you do know the definition, and are willing to do a small amount of computation, you can do a sort of update with this new information to get better heuristics, just like in Terry Tao’s post linked above. The restriction on the amount of computation is necessary to make this idea useful; if you had unlimited computational resources, you would know everything about the primes already.
It’s exciting to see in number theory lots of work on finding good priors for these sorts of things, another good example is with the Cohen-Lenstra heuristics which are some conjectures ultimately based on the prior probability of a generic group of some kind having a certain property (sorry, I don’t remember very well). It seems like work dealing with the limited computational resources mostly has yet to be done, but there might be lots of good ideas in computational complexity theory (not my domain of expertise). I think that having this kind of theory will also be important to FAI research, since it will need to reason correctly about math with limited resources.
Anyway, I think you are right that Euler had enough evidence to be confident of his solution to the Basel problem. I hope that a solid theory for this sort of reasoning is found soon, and I’m glad to see that other people are thinking about it.