GPT-8 may not be ASI

Imagine the gap between GPT-2 and GPT-3. Imagine the gap between GPT-3 and GPT-4. Let’s suppose GPT-5 has the same gap with GPT-4, and GPT-6 has the same gap with GPT-5, and so on till GPT-8.

It is still entirely possible that GPT-8 is not better than the best of humanity at all possible tasks. There can still exist multiple tasks at which the best humans beat GPT-8.

One way to build this intuition is the following thought experiment: Try imagining a GPT-5 that makes headway in GPT4′s weaknesses, but has its own weaknesses. Try imagining a GPT-6 that makes headway in GPT-5′s weaknesses, but has its own weaknesses. Iterate till you get GPT-8.

Here’s one run of this thought experiment:

My thought experiments focusses on mathematics, but you could also pick a natural language example and run your own thought experiment instead.

GPT4 has high error rate per reasoning step, and sequentially chaining many such reasoning steps blows up error to 1. GPT4 cannot accurately answer square root of 1234.

Let’s suppose GPT-5 can look at sequentially chained reasoning steps that are likely to occur in training, and for those chains it is able to perform the entire chain with low error. But if it needs to join separate reasoning steps in a novel manner, it again has high error. GPT5 answers square root of 1234 correctly because it has seen previous square root examples and internalises all the steps of the square root algorithm (in a way that doesn’t add up error with each step). But it cannot answer (1234 + 5678^(1/2))^(1/3).

Let’s suppose GPT-6 can do the same thing that GPT-5 does, but first it can transform the input space into a variety of other spaces. For instance it can transform numbers into a space that naturally does binomial expansions (including of fractional exponents) and it can transform symbols into a space that naturally does arithmetic using a FIFO stack and it can transform symbols into a space that does first-order logic. Let’s suppose however that GPT-6 now requires a lot of data to get this to work though. For instance to make it solve all possible arithmetic expressions of length 50 symbols, it needs 1 trillion examples of length 50 symbols. So it can answer (1234 + 5678^(1/2))^(1/3) but it can’t prove a statement as complicated as the fundamental theorem of algebra, assuming the proof does not exist verbatim in its training data.

Let’s suppose GPT-7 can take its output as input in a way that guarantees its output is slightly better than its input. You can give it a statement to prove, it will open a search tree over millions of inferences. Every time you perform an inference you take the output and give it back as input for the next inference. Assume however that there are certain regions (in the space of all possible proofs) that it is fast at searching, and certain regions it is slow at searching. So it can prove fundamental theorem of algebra, it can solve four colour theorem and create new branches of mathematics, but for some unknown reason it gets completely stuck on geometric proofs. So it cannot do any of the IMO goemetry problems, for example.

Let’s suppose GPT-7 finds an ML inference technique that allows you to intelligently cache inferences and speedup the inference of prompts that are similar but not identical to earlier prompts. This allows you to now run GPT-7 inferences 10^9 times faster on average although worst case time is same. This faster version of GPT-7 plus some more training data and compute is called GPT-8. GPT-8 is much faster than human mathematicians at all branches of mathematics. Where it lacks in “intuition” of genius mathematicians, it makes up for in raw speed at a heuristics-based search no one fully understands. However it occasionally still hallucinates because it’s fundamentally based on GPT-5′s technique even now. For example if it writes a 200-page book on a branch of mathematics it has invented, there may be a hallucination on page 154 that invalidates the next 46 pages of work. Progress in mathematics is entirely dependent on the speed at which human mathematicians can wrap their heads around the pages of proofs GPT-8 throws at them. GPT-8 can also assist with natural language explanations that help build intuitions in a way the human brain can understand them. But often it fails because its intuitions are alien and it doesn’t know a way to translate that into an intuition the human brain can grasp. GPT-8 still cannot solve half of the Clay Millenium problems, but human mathematicians can use GPT-8′s help, build their own intuition and solve these problems.

Dyson spheres and self-replicating nanofactories still look quite far off in the world GPT-8 exists, although people believe we’ll get there one day.