In part 2, I sort of glossed over the technical stuff, but I was not talking about making up political dimensions like “more anti-war” and rating the answers to poll questions by hand. That is way too arbitrary for my taste. I’m talking about plain old dimensionality reduction. I had something like PCA in mind (if we were careful we might use a different method, but this is just illustrative.)
If you don’t know about Principal Components Analysis, it’s an important notion.
The principle is that if you decide a priori what the “coordinates” are, you might pick wrong. You might not explain the variability in the data very well. Amazon.com doesn’t have a pre-set category called “horror” and recommend you horror movies based on the fact that you’ve watched other horror movies. Amazon gauges “similarity” based on coordinates that arise naturally from the data (and maybe don’t easily correspond to a property that can be given an English name.)
Maybe I’ll write a top-level post explaining this sometime, if it isn’t common knowledge.
In part 2, I sort of glossed over the technical stuff, but I was not talking about making up political dimensions like “more anti-war” and rating the answers to poll questions by hand. That is way too arbitrary for my taste. I’m talking about plain old dimensionality reduction. I had something like PCA in mind (if we were careful we might use a different method, but this is just illustrative.)
If you don’t know about Principal Components Analysis, it’s an important notion.
Wiki
Tutorial with a practical example
Another intro
The principle is that if you decide a priori what the “coordinates” are, you might pick wrong. You might not explain the variability in the data very well. Amazon.com doesn’t have a pre-set category called “horror” and recommend you horror movies based on the fact that you’ve watched other horror movies. Amazon gauges “similarity” based on coordinates that arise naturally from the data (and maybe don’t easily correspond to a property that can be given an English name.)
Maybe I’ll write a top-level post explaining this sometime, if it isn’t common knowledge.
Principal component analysis of UK political views, from a few years back: http://politicalsurvey2005.com/themap.pdf
Thanks—that’s roughly what I was talking about.
A pdf of the nature intro is here.