Hi. Your idea seems to be, that if probabilistic inference isn’t able to recognize the theorems of a formal system, then we shouldn’t expect language models to arrive at truth in general.
I don’t know if anyone has studied statistical regularities in theorems. You could ask on the “foundations of mathematics” list. But I think this is not a particularly effective critique of the truth capabilities of language models.
On the one hand, the epistemology of theorems is quite different to the epistemology of worldly facts, like historical, subjective, or practical facts, and yet it’s the latter that people are mostly concerned with.
On the other hand, it’s not clear that human mathematicians are inherently better than language models at spotting possible truths, since this involves intuition and insight, and that’s all about heuristics—mental rules of thumb that generate plausible hypotheses. And language models are clearly heuristic rather than deductive in how they generate their responses. So there’s no reason they can’t get as good as human mathematicians, or better… And in fact the same applies to much of the worldly truths as well—so much of what we decide to believe has a heuristic origin, rather than arising from definite knowledge.
If there is a serious difference between humans and language models, I’d say it hinges on consciousness. The human sense of what is true, is bound up with the mysteries of awareness, being, and the awareness of being. Conscious experience has a subjective and an objective pole, science focuses on the objective pole, and various scientific reductionisms try to reduce subjective ontology to entities from physics, computer science, or mathematics. In the distant past I wrote against this on Less Wrong, as well as advocating for the kind of quantum mind theory which says that consciousness is possibly grounded in what physicists call entanglement; something which implies that AI on a classical computer wouldn’t be conscious.
The relationship between AIs, humans, belief, and truth is very multifaceted. The potentials run from very good to very bad. I’ve tried to sketch why I think truth in formal systems is not very relevant to the enthusiasm for language models, and where I think there’s a genuine ontological difference between AI and humans. I’ll also add that there is some work on identifying what a language model actually “believes” (spotted via June Ku’s Twitter), which could perhaps be used to train a language model to care more about truth.
Hi. Your idea seems to be, that if probabilistic inference isn’t able to recognize the theorems of a formal system, then we shouldn’t expect language models to arrive at truth in general.
I don’t know if anyone has studied statistical regularities in theorems. You could ask on the “foundations of mathematics” list. But I think this is not a particularly effective critique of the truth capabilities of language models.
On the one hand, the epistemology of theorems is quite different to the epistemology of worldly facts, like historical, subjective, or practical facts, and yet it’s the latter that people are mostly concerned with.
On the other hand, it’s not clear that human mathematicians are inherently better than language models at spotting possible truths, since this involves intuition and insight, and that’s all about heuristics—mental rules of thumb that generate plausible hypotheses. And language models are clearly heuristic rather than deductive in how they generate their responses. So there’s no reason they can’t get as good as human mathematicians, or better… And in fact the same applies to much of the worldly truths as well—so much of what we decide to believe has a heuristic origin, rather than arising from definite knowledge.
If there is a serious difference between humans and language models, I’d say it hinges on consciousness. The human sense of what is true, is bound up with the mysteries of awareness, being, and the awareness of being. Conscious experience has a subjective and an objective pole, science focuses on the objective pole, and various scientific reductionisms try to reduce subjective ontology to entities from physics, computer science, or mathematics. In the distant past I wrote against this on Less Wrong, as well as advocating for the kind of quantum mind theory which says that consciousness is possibly grounded in what physicists call entanglement; something which implies that AI on a classical computer wouldn’t be conscious.
The relationship between AIs, humans, belief, and truth is very multifaceted. The potentials run from very good to very bad. I’ve tried to sketch why I think truth in formal systems is not very relevant to the enthusiasm for language models, and where I think there’s a genuine ontological difference between AI and humans. I’ll also add that there is some work on identifying what a language model actually “believes” (spotted via June Ku’s Twitter), which could perhaps be used to train a language model to care more about truth.