So, I like this, but I’m still not sure I understand where features come from.
Say I’m an AI, and I’ve observed a bunch of sensor data that I’m representing internally as the points (6.94, 3.96), (1.44, -2.83), (5.04, 1.1), (0.07, -1.42), (-2.61, -0.21), (-2.33, 3.36), (-2.91, 2.43), (0.11, 0.76), (3.2, 1.32), (-0.43, -2.67).
The part where I look at this data and say, “Hey, these datapoints become approximately conditionally independent if I assume they were generated by a multivariate normal with mean (2, -1), and covariance matrix [[16, 0], [0, 9]][1]; let me allocate a new concept for that!” makes sense. (In the real world, I don’t know how to write a program to do this offhand, but I know how to find what textbook chapters to read to tell me how.)
But what about the part where my sensor data came to me already pre-processed into the list of 2-tuples?—how do I learn that? Is it just, like, whatever transformations of a big buffer of camera pixels let me find conditional independence patterns probably correspond to regularities in the real world? Is it “that easy”??
In the real world, I got those numbers from the Python expression ', '.join(str(d) for d in [(round(normal(2, 4), 2), round(normal(-1, 3), 2)) for _ in range(10)]) (using scipy.random.normal).
Is it just, like, whatever transformations of a big buffer of camera pixels let me find conditional independence patterns probably correspond to regularities in the real world? Is it “that easy”??
Roughly speaking, yes.
Features are then typically the summary statistics associated with some abstraction. So, we look for features which induce conditional independence patterns in the big buffer of camera pixels. Then, we look for higher-level features which induce conditional independence between those features. Etc.
It’s funny that you should mention this, because I’ve considered working on a machine learning system for image recognition using this principle. However, I don’t think this is necessarily all of it. I bet we come pre-baked with a lot of rules for what sorts of features to “look for”. To give an analogy to machine learning, there’s an algorithm called pi-GAN, which comes pre-baked with the assumption that pictures originate from 3D scenes, and which then manages to learn 3D scenes from the 2D images it is trained with. (Admittedly only when the images are particularly nice.)
So, I like this, but I’m still not sure I understand where features come from.
Say I’m an AI, and I’ve observed a bunch of sensor data that I’m representing internally as the points
(6.94, 3.96), (1.44, -2.83), (5.04, 1.1), (0.07, -1.42), (-2.61, -0.21), (-2.33, 3.36), (-2.91, 2.43), (0.11, 0.76), (3.2, 1.32), (-0.43, -2.67).The part where I look at this data and say, “Hey, these datapoints become approximately conditionally independent if I assume they were generated by a multivariate normal with mean
(2, -1), and covariance matrix[[16, 0], [0, 9]][1]; let me allocate a new concept for that!” makes sense. (In the real world, I don’t know how to write a program to do this offhand, but I know how to find what textbook chapters to read to tell me how.)But what about the part where my sensor data came to me already pre-processed into the list of 2-tuples?—how do I learn that? Is it just, like, whatever transformations of a big buffer of camera pixels let me find conditional independence patterns probably correspond to regularities in the real world? Is it “that easy”??
In the real world, I got those numbers from the Python expression
', '.join(str(d) for d in [(round(normal(2, 4), 2), round(normal(-1, 3), 2)) for _ in range(10)])(usingscipy.random.normal).Roughly speaking, yes.
Features are then typically the summary statistics associated with some abstraction. So, we look for features which induce conditional independence patterns in the big buffer of camera pixels. Then, we look for higher-level features which induce conditional independence between those features. Etc.
This gave me a blog story idea!
“Feature Selection”
It’s funny that you should mention this, because I’ve considered working on a machine learning system for image recognition using this principle. However, I don’t think this is necessarily all of it. I bet we come pre-baked with a lot of rules for what sorts of features to “look for”. To give an analogy to machine learning, there’s an algorithm called pi-GAN, which comes pre-baked with the assumption that pictures originate from 3D scenes, and which then manages to learn 3D scenes from the 2D images it is trained with. (Admittedly only when the images are particularly nice.)