By myopic I mean https://www.lesswrong.com/tag/myopia — that it was trained to predict the next token and doesn’t get much lower loss from having goals about anything longer-term than predicting the next token correctly.
I assume the weights are frozen, I’m surprised to see this as a question.
Some quick replies from the top of my head: If GPT-7 has a much larger context window; or if there are kinds of prompts the dynamic converges to that aren’t too long; and you get an AGI that’s smart and goal-oriented and needs to spend some of the space that it has to support its level (or it naturally happens, because the model continues to output what an AGI that smart would be doing), and if how smart an AGI simulated by that LLM might be isn’t capped at some low level, I don’t think there’s any issue with it using notes until it can has access to something outside, that allows it to be more of a AutoGPT with external memory and everything. If it utilises the model’s knowledge, it might figure out what text it can output that hacks the server where the text is stored and processed; or it can understand humans and design a text that hacks their brains when they look at it.
By myopic I mean https://www.lesswrong.com/tag/myopia — that it was trained to predict the next token and doesn’t get much lower loss from having goals about anything longer-term than predicting the next token correctly.
I assume the weights are frozen, I’m surprised to see this as a question.
Some quick replies from the top of my head: If GPT-7 has a much larger context window; or if there are kinds of prompts the dynamic converges to that aren’t too long; and you get an AGI that’s smart and goal-oriented and needs to spend some of the space that it has to support its level (or it naturally happens, because the model continues to output what an AGI that smart would be doing), and if how smart an AGI simulated by that LLM might be isn’t capped at some low level, I don’t think there’s any issue with it using notes until it can has access to something outside, that allows it to be more of a AutoGPT with external memory and everything. If it utilises the model’s knowledge, it might figure out what text it can output that hacks the server where the text is stored and processed; or it can understand humans and design a text that hacks their brains when they look at it.