Good post, but also there might be enough inertia to using the word “feature” in different contexts that it’s hard to stop. Honestly the in-the-world vs in-the-model distinction my be the less confusing of the two common distinctions, because in both cases a feature is a part of a decomposition of the whole into parts that can be composed with each other. The more subtle one to keep straight is the distinction between features as things found by their local statistical properties vs. features as things found by their impact on the entire computation.
Good post, but also there might be enough inertia to using the word “feature” in different contexts that it’s hard to stop. Honestly the in-the-world vs in-the-model distinction my be the less confusing of the two common distinctions, because in both cases a feature is a part of a decomposition of the whole into parts that can be composed with each other. The more subtle one to keep straight is the distinction between features as things found by their local statistical properties vs. features as things found by their impact on the entire computation.