I’m assuming “natural abstraction” is also a scalar property. Reading this paragraph, I refactored the concept in my mind to “some abstractions tend to be cheaper to abstract than others. agents will converge to using cheaper abstractions. Many cheapness properties generalize reasonably well across agents/observation-systems/environments, but, all of those could in theory come apart.”
And the Strong NAH would be “cheap-to-abstract-ness will be very punctuated, or something” (i.e. you might expect less of a smooth gradient of cheapnesses across abstractions)
The way I think of it, it’s not quite that some abstractions are cheaper to use than others, but rather:
One can in-principle reason at the “low(er) level”, i.e. just not use any given abstraction. That reasoning is correct but costly.
One can also just be wrong, e.g. use an abstraction which doesn’t actually match the world and/or one’s own lower level model. Then predictions will be wrong, actions will be suboptimal, etc.
Reasoning which is both cheap and correct routes through natural abstractions. There’s some degrees of freedom insofar as a given system could use some natural abstractions but not others, or be wrong about some things but not others.
I’m assuming “natural abstraction” is also a scalar property. Reading this paragraph, I refactored the concept in my mind to “some abstractions tend to be cheaper to abstract than others. agents will converge to using cheaper abstractions. Many cheapness properties generalize reasonably well across agents/observation-systems/environments, but, all of those could in theory come apart.”
And the Strong NAH would be “cheap-to-abstract-ness will be very punctuated, or something” (i.e. you might expect less of a smooth gradient of cheapnesses across abstractions)
The way I think of it, it’s not quite that some abstractions are cheaper to use than others, but rather:
One can in-principle reason at the “low(er) level”, i.e. just not use any given abstraction. That reasoning is correct but costly.
One can also just be wrong, e.g. use an abstraction which doesn’t actually match the world and/or one’s own lower level model. Then predictions will be wrong, actions will be suboptimal, etc.
Reasoning which is both cheap and correct routes through natural abstractions. There’s some degrees of freedom insofar as a given system could use some natural abstractions but not others, or be wrong about some things but not others.