If the model that is used as a Microscope AI does not use any optimization (search), how will it compute the probability that, say, Apple’s engineers will overcome a certain technical challenge?
That’s a good question. Perhaps it does make use of optimization but the model still has an overall passive relationship to the world compared to an active mesa-optimizer AI. I’m thinking about the difference between say, GPT-3 and the classic paperclip maximizer or other tiling AI.
This is just my medium-confidence understanding and may be different from what Evan Hubinger meant in that quote.
I believe it would look like Microscope AI.
If the model that is used as a Microscope AI does not use any optimization (search), how will it compute the probability that, say, Apple’s engineers will overcome a certain technical challenge?
That’s a good question. Perhaps it does make use of optimization but the model still has an overall passive relationship to the world compared to an active mesa-optimizer AI. I’m thinking about the difference between say, GPT-3 and the classic paperclip maximizer or other tiling AI.
This is just my medium-confidence understanding and may be different from what Evan Hubinger meant in that quote.