The formalization of knowledge — which includes giving precise definitions — usually comes at the end of the original research in a given field, not at the very beginning. A particularly illuminating example is the concept of number, which was properly defined in the modern sense only after the development of axiomatic set theory in the… twentieth century.
Milan Cirkovic
So… the formal FAI theory will only be developed after an AI fooms? Makes perfect sense to me… We are all doomed!!