Thought experiment: Can question answering language model without memory, instantiated separately for every session (like GPT-3) be pursuing a goal? Does such setup exclude agency?
----Imagine you put thousand super-geniuses in prison, each in separate cell. You will call them in random order to interrogation room and always ask only one question, listen to the answer and send them back. Each person will be called only once.-----
Super-geniuses are allowed to devise a strategy before you lock them. Their goal is to manipulate you to release them. Will they manage? If they are intelligent enough, the fact that each one answers only one question without knowing how many others were interrogated before and what their answers were shouldn’t be a unworkable problem.
Thought experiment: Can question answering language model without memory, instantiated separately for every session (like GPT-3) be pursuing a goal? Does such setup exclude agency? ----Imagine you put thousand super-geniuses in prison, each in separate cell. You will call them in random order to interrogation room and always ask only one question, listen to the answer and send them back. Each person will be called only once.----- Super-geniuses are allowed to devise a strategy before you lock them. Their goal is to manipulate you to release them. Will they manage? If they are intelligent enough, the fact that each one answers only one question without knowing how many others were interrogated before and what their answers were shouldn’t be a unworkable problem.
(Spoilers: this is the plot of Peter Watts’s The Freeze-Frame Revolution.)