The human knows the rules and the win condition. The optimisation algorithm doesn’t, for the same reason that evolution doesn’t “know” what dying is: neither are the types of entities to which you should ascribe knowledge.
Suppose you have a computer program that gets two neural networks, simulates a game of go between them, determines the winner, and uses the outcome to modify the neural networks. It seems to me that this program has a model of the ‘go world’, i.e. a simulator, and from that model you can fairly easily extract the rules and winning condition. Do you think that this is a model but not a mental model, or that it’s too exact to count as a model, or something else?
I’d say that this is too simple and programmatic to be usefully described as a mental model. The amount of structure encoded in the computer program you describe is very small, compared with the amount of structure encoded in the neural networks themselves. (I agree that you can have arbitrarily simple models of very simple phenomena, but those aren’t the types of models I’m interested in here. I care about models which have some level of flexibility and generality, otherwise you can come up with dumb counterexamples like rocks “knowing” the laws of physics.)
As another analogy: would you say that the quicksort algorithm “knows” how to sort lists? I wouldn’t, because you can instead just say that the quicksort algorithm sorts lists, which conveys more information (because it avoids anthropomorphic implications). Similarly, the program you describe builds networks that are good at Go, and does so by making use of the rules of Go, but can’t do the sort of additional processing with respect to those rules which would make me want to talk about its knowledge of Go.
On that definition, how does one train an AlphaZero-like algorithm without knowing the rules of the game and win condition?
The human knows the rules and the win condition. The optimisation algorithm doesn’t, for the same reason that evolution doesn’t “know” what dying is: neither are the types of entities to which you should ascribe knowledge.
Suppose you have a computer program that gets two neural networks, simulates a game of go between them, determines the winner, and uses the outcome to modify the neural networks. It seems to me that this program has a model of the ‘go world’, i.e. a simulator, and from that model you can fairly easily extract the rules and winning condition. Do you think that this is a model but not a mental model, or that it’s too exact to count as a model, or something else?
I’d say that this is too simple and programmatic to be usefully described as a mental model. The amount of structure encoded in the computer program you describe is very small, compared with the amount of structure encoded in the neural networks themselves. (I agree that you can have arbitrarily simple models of very simple phenomena, but those aren’t the types of models I’m interested in here. I care about models which have some level of flexibility and generality, otherwise you can come up with dumb counterexamples like rocks “knowing” the laws of physics.)
As another analogy: would you say that the quicksort algorithm “knows” how to sort lists? I wouldn’t, because you can instead just say that the quicksort algorithm sorts lists, which conveys more information (because it avoids anthropomorphic implications). Similarly, the program you describe builds networks that are good at Go, and does so by making use of the rules of Go, but can’t do the sort of additional processing with respect to those rules which would make me want to talk about its knowledge of Go.