I’m imagining a scenario in which OpenAI etc. continue to scale up their language models, and eventually we get GPT-6, which has the following properties:
--It can predict random internet text better than the best humans
--It can answer correctly questions which seem to require long chains of reasoning to answer
--With appropriate prompts it can write novel arguments, proofs, code, etc. of quality about equal to the stuff it has read on the internet. (The best stuff, if the prompt is designed correctly)
--With appropriate prompts it can give advice about arbitrary situations, including advice about strategies and plans. Again, the advice is about as good as the stuff it read on the internet, or the best stuff, if prompted correctly.
--It costs $200 per page of output, because just running the model requires a giant computing cluster.
My question is, how does this transform the world? I have the feeling that the world would be transformed pretty quickly. At the very least, the price of running the model would drop by orders of magnitude over the next few years due to algorithmic and hardware improvements, and then we’d see lots of jobs getting automated. But I’m pretty sure stuff would go crazy even before then. How?
(CONTEXT: I’m trying to decide whether “Expensive AGI” is meaningfully different from the usual AGI scenarios. If we get AGI but it costs $200 per page instead of $2, and thus isn’t economically viable for most jobs, does that matter? EDIT: What if it costs $2,000 or $20,000 per page? Do things go FOOM soon even in that case?)
$200 per page of quality proof output is super cheap—typical Fermi calculation shows a typical mathematician cost about $100 000 a year, and output about 100* pages of peer-reviewed papers per years at best, so about $1000 per page for pure maths (applied maths are maybe half as expensive ?).
So 1st consequence: every single theoretical scientist get fired. Computer scientist are more expensive and get fired even earlier. Also journalists, lawyers, accountants and basically any job that requires high writing skills.
*wages may be about $200 000 and output as high as 200 pages depending of country/talent/field but the order of magnitude is the same.
** first post here ;)
Good point—some humans make more than $200 per page of output. Maybe I should have said $2000, or maybe not that it could produce the best-quality stuff, but instead just average-quality stuff.
I believe the central impact will be a powerful compression of knowledge and a flood of legibility, which will be available to institutions and leadership first. Examples include:
Speechwriting
Report summarization
Report generation
Even the higher number, like $20,000 per page, is a good deal for something like Wikipedia, where the page is available to millions of readers, or for things like the Stanford Encyclopedia of Philosophy. This will have a big impact on:
Online encyclopedias
Online textbooks
While this could easily be used to generate high-quality propaganda, I feel like it still weighs much more heavily in favor of the truth. This is because bullshit’s biggest advantage is that is is fast, cheap and easy to vary, whereas reality is inflexible and we comprehend it slowly. But under the proposed conditions, advanced bullshit and the truth cost the same amount, and have a similar speed. This leaves reality’s inflexible pressure on every dimension of every problem a decisive advantage in favor of the truth. This has a big impact on things like:
Any given prediction
Project proposals
Especially if it is at the lower end of the price scale, it becomes trivial to feed it multiple prompts and get multiple interpretations of the same question. This will give us a lot of information both in terms of compression and also in terms of method, which will cause us to be able to redirect resources into the most successful methods, and also to drop inefficient ones. I further expect this to be very transparent very quickly, though mechanisms like:
Finance
Sports betting
It will see heavy use by the intelligence community. A huge problem we have in the United States is our general lack of language capability; for example if GPT-6 knows Mandarin as well as any Mandarin speaker, and translates to English as well as any translator, then suddenly we get through the bottleneck and gain access to good information about Chinese attitudes. I expect this same mechanism will make foreign investment much more attractive almost universally, since domestic and foreign firms will now be working on an almost level playing field in any country with widespread internet access. If this prediction holds, I expect a large boom in investment in otherwise underdeveloped countries, because the opportunities will finally be legible.
Another interesting detail is that if GPT-6 can provide the best summaries of the available knowledge, this means that most of the world’s institutions will then be working from a much more uniform knowledge base than we do currently. My initial reaction was that this is clearly for the best because the biggest roadblock to coordination is getting on the same page with the other stakeholders, but it also occurs to me that it makes transparent to everyone the cases where certain stakeholders have an untenable position. I suspect this in turn makes it more likely that some parties get the screws put to them, and further when they a) understand their own position and b) understand that everyone else understands it, they are more likely to try something radical to shift the outcome. Consider North Korea, for example.
Someone retrains the model using reinforcement learning to be more of an agent. Maybe a chatbot that tries to convince people of things, or give good advice, or make good predictions, or some combination thereof. This unlocks its hidden intelligence, so to speak, since it no longer thinks it is predicting other people’s text. It’s now basically a human-level AGI; it’s expensive, but if they make it bigger and train it for longer maybe they can make a new version which is superhuman, and then things will go off the rails, because a superhuman AGI is well worth $20,000 per page of output.
However, this would take at least a few more months, probably more than a year, to achieve. I suspect that a lot of important things would happen sooner than that.
This means it can actually do long chains of reasoning by manipulating what’s in his short term memory similar to how humans do long chains of reasoning by operating on what’s in their working memory.
The ability to do reasoning the means that the quality isn’t very dependent on what can be found on the internet.
An AGI that’s human level for the average problem likely has problems where it outperforms humans.
The AI described isn’t trying to outperform humans, its been optimised to imitate humans. Of course, there is a potential for mesa-optimization, but I don’t think that would lead to a system that produced better text. (It might lead to the system producing strange or subtly manipulative text.)
The AGI has access to a bunch of background knowledge that humans don’t have. No human mathematician will have read as much different math tools as the AGI.
There’s the potential for the working memory to be comparatively larger then humans working memory in some cases. The AGI has a more explicit ability to add and remove items from it’s working memory.
Even if the AGI doesn’t try to outperform humans it’s different enough from humans that it won’t have the same performance at every task as humans and likely be worse at some tasks while being better at others.
Take arithmetic. Lets assume that given the computational resources available, it would be utterly trivial to do perfect arithmetic. Lets also assume that the training data was written by people who were somewhat innumerate. Lets say that many of the arithmetical statements that appear in the training dataset are wrong.
You give it the prompt “2+2=”. The training data contained “2+2=7″ as often as “2+2=4”. The AI is only being selected towards the sort of text strings that exist in the training dataset. It has no concept that by “+” you mean addition and not something else.
Of course, if humans give the correct answer 10% of the time, and 90% of the time give a wrong answer, but any particular wrong answer appears <1% of the time, you could find the right answer by taking the mode.
If a lot of the trainings example say “2+2=7” then of cause the AI will not think that + means addition because it doesn’t. If however people use + to mean addition GPT3 is already capable enough to learn the concept and use it to add numbers that aren’t in it’s training corpus.
To have human level cognition you need the ability to use multiple concepts together in a new way. Knowing more mathematical concepts then the average mathematician might lead to better performance given that mathematical proofs are a lot about needing to know the concepts that are required for a given proof.
I also think that for GPT-x to reach AGI-hood it will need a large enough attention field to use part of that attention field as memory which means it can do reasoning in additional ways.
Yes, but it will still be about as good as its training corpus.
One way of looking at this is that GPT-X is trying to produce text that looks just like human written text. Given two passages of text, there should be no easy way to tell which was written by a human, and which wasn’t.
GPT-X has expertise in all subjects, in a sense. Each time it produces text, it is sampling from the distribution of human competence. Detailed information about anteaters is in there somewhere, every now and again, it will sample an expert on them, but most of the time it will act like a person who doesn’t know much about anteaters.
We can end up with an intelligence explosion via automated ML research. One of the tasks that could be automated by the language model is “brainstorming novel ML ideas”. So you’ll be able to pay $200 and get a text, that could have been written by a brilliant ML researcher, containing novel ideas that allow you to create a more efficient/capable language model. (Though I expect that this specific approach won’t be competitive with fully automated approaches that do stuff like NAS.)
‘Predicting random text on the internet better than a human’ already qualifies it as superhuman, as dirichlet-to-neumann pointed out. If you look at any given text, there’s a given ratio of cognitive work needed to produce the text, per word-count. “Superhuman” only requires asking it to replicate the work of multiple people collaborating together, or processes which need a lot of human labour like putting together a business strategy or writing a paper. Assuming it’s mediocre in some aspects, the clearest advantage GPT-6 would have would be an interdisciplinary one—pooling together domain knowledge from disparate areas to produce valuable new insights.
Some tasks require much less than a page of output: —Brainstorming ideas, e.g. for product/character names, plot twists, etc. —Providing advice, e.g. as a therapist or financial advisor or lawyer —Answering questions, e.g. as a financial advisor or lawyer Perhaps there would be an economic niche for automating away some of these tasks.
Put it in the hands of inventors and researchers writing up business plans (with relevant metadata and project/company outcomes), and in the hands of VC, CVC, and other investors (including internally at larger companies). This would eliminate an enormous amount of friction, and energy wasted on both easily foreseeable mistakes and charades all parties have to play into today. That alone is sufficient to increase the rate of long-term economic and technological growth. By increasing ROR on investment, it will also increase total investment economy-wide. I’d expect this to become very common even at >$20k/page, since the expected values for success can be 8-9 figures and the vast majority of companies and projects fail.
Based on “Why Tool AIs Want to Be Agent AI’s” by Gwern, I would expect an AGI level GPT-6 to self improve and become a world gobbling AI.
The moment it gets a hint that it could answer better by getting (unknown bit of data from the Internet, extra memory, some other resource), the software’s own utility function will push the machine in that direction.
OK, but in this case I’m trying to imagine something that’s not significantly smarter than humans. So it probably can’t think of any self-improvement ideas that an AI scientist wouldn’t have thought of already, and even if it did, it wouldn’t have the ability to implement them without first getting access to huge supercomputers to re-train itself. Right?
I worry that I’m splitting hairs now because it seems that the AI only needs to be clever enough to generate the following in response to a query :
The answer to your question will be provided more quickly if you provide 1 GB of RAM. (rinse and repeat until we get to an AI box)