A system capable of reasoning about optimization is likely also capable of reusing that same machinery to do optimization itself
I’m confused about this. I tried substituting different words for “optimisation”:
“A system capable of reasoning about photosynthesis is likely also capable of reusing that same machinery to do photosynthesis itself.” Nope.
“A system capable of reasoning about arithmetic is likely also capable of reusing that same machinery to do arithmetic itself”. Maybe? The rules of arithmetic can be reused, but the machinery to reason abstractly about arithmetic is probably different to the machinery to run a specific calculation, especially with a learned model with lots of free parameters, like a neural network.
Maybe optimisation is not like the above examples because it is so generic? Or I misunderstood the claim.
I’m confused about this. I tried substituting different words for “optimisation”:
“A system capable of reasoning about photosynthesis is likely also capable of reusing that same machinery to do photosynthesis itself.” Nope.
“A system capable of reasoning about arithmetic is likely also capable of reusing that same machinery to do arithmetic itself”. Maybe? The rules of arithmetic can be reused, but the machinery to reason abstractly about arithmetic is probably different to the machinery to run a specific calculation, especially with a learned model with lots of free parameters, like a neural network.
Maybe optimisation is not like the above examples because it is so generic? Or I misunderstood the claim.