I think that you have a point, but that arguably there is some goalpost moving going on somewhere in here.
Say, some computer system recognises trees, and has something in a layer of its mathematics that roughly corresponds to its idea of tree. Maybe that idea of a tree is no further removed from the real object than my hazy thought-cloud that goes with the word “tree”—but so what? When talking about how interpretable something is to me the question is not one of distance from reality, but the distance to my concepts.
I drew a random sketch to clarify my point. Lets say each node represents “tree”. So the one on the right is a real tree, the bottom one the English word tree, the top the program’s idea of a tree. (We can suppose that my human idea of a tree is another node that connects with human language and reality). Interpretability is (to me) the line on the left, while much of your post is about the difficulties with the line on the bottom right.
To the extent that a system’s concepts match my concepts but not reality, I would expect that to be some form of deception; the system is playing to mistakes I am making.
I think that you have a point, but that arguably there is some goalpost moving going on somewhere in here.
Say, some computer system recognises trees, and has something in a layer of its mathematics that roughly corresponds to its idea of tree. Maybe that idea of a tree is no further removed from the real object than my hazy thought-cloud that goes with the word “tree”—but so what? When talking about how interpretable something is to me the question is not one of distance from reality, but the distance to my concepts.
I drew a random sketch to clarify my point. Lets say each node represents “tree”. So the one on the right is a real tree, the bottom one the English word tree, the top the program’s idea of a tree. (We can suppose that my human idea of a tree is another node that connects with human language and reality). Interpretability is (to me) the line on the left, while much of your post is about the difficulties with the line on the bottom right.
To the extent that a system’s concepts match my concepts but not reality, I would expect that to be some form of deception; the system is playing to mistakes I am making.