Someone retrains the model using reinforcement learning to be more of an agent. Maybe a chatbot that tries to convince people of things, or give good advice, or make good predictions, or some combination thereof. This unlocks its hidden intelligence, so to speak, since it no longer thinks it is predicting other people’s text. It’s now basically a human-level AGI; it’s expensive, but if they make it bigger and train it for longer maybe they can make a new version which is superhuman, and then things will go off the rails, because a superhuman AGI is well worth $20,000 per page of output.
However, this would take at least a few more months, probably more than a year, to achieve. I suspect that a lot of important things would happen sooner than that.
Someone retrains the model using reinforcement learning to be more of an agent. Maybe a chatbot that tries to convince people of things, or give good advice, or make good predictions, or some combination thereof. This unlocks its hidden intelligence, so to speak, since it no longer thinks it is predicting other people’s text. It’s now basically a human-level AGI; it’s expensive, but if they make it bigger and train it for longer maybe they can make a new version which is superhuman, and then things will go off the rails, because a superhuman AGI is well worth $20,000 per page of output.
However, this would take at least a few more months, probably more than a year, to achieve. I suspect that a lot of important things would happen sooner than that.