One thing we do know is that if we project all the opinions onto a set of dimensions that we like—for example, the old standby of liberal/conservative and authoritarian/libertarian—we can start to measure “political diversity.” People who describe themselves as Democrats, for example, span a much wider range of views than people who describe themselves as “Republicans.” Implicitly, we’re putting a Euclidean distance on a high-dimensional space, projected onto a few dimensions.
Trouble is, any concrete definition of this distance will be arbitrary. Even with a single dimension, it is only possible to construct an ordinal scale, and there is no objective way to define distance. With multiple dimensions, there is the additional problem of how to make different dimensions commensurable. By varying these arbitrary parameters, any such calculation can be manipulated to reach very different conclusions.
An assertion that people in one group have a wider range of views than those in another can be defended only insofar as common-sense clearly leads to such a conclusion. If this is not the case, then the answer given by all this flashy math will be determined by your own preconceptions you put into the model, not some objective truth.
If we used some kind of resource distribution to acquire the information; i.e. “you have a budget how much do you spend on what” you would (in addition to learning how little people know about economics) have some pretty solid numbers to work with in defining a metric.
The issues of budget allocation are only a subset of people’s political opinions. But even with concrete budget numbers, the metric still cannot be objective.
Suppose you have three people who advocate, respectively, a $700B military budget, a $350B military budget, and a $0 budget (i.e. complete abolition of the armed forces). Clearly, common sense tells us that the first two people have a large difference of opinion, but it pales in comparison with the extremism of the third one’s position, and it makes no sense to claim that the distances 1-2 and 2-3 are equal, even though the difference in numbers is the same. So again you have to introduce some arbitrary scaling to define this distance.
Moreover, the differences of opinion on different budget items cannot be compared directly based on just the amounts of money allocated. The budged of the FDA is only a few billion, but abolishing the FDA is a far more radical proposition that cutting back, say, the military budget by ten times what the FDA now gets. Again, you have to introduce some arbitrary criteria to compare these numbers.
So, ultimately, you’ll end up producing arbitrary numbers no matter what.
Both of your examples are ultimately arguments in favor of reasoning about ratios of budgets: going from $350B to $700B is a 100% increase, while going from $0 to $350B isn’t even defined. Perhaps $175B and $350B would be at a similar distance from each other.
Similarly, taking away a few billion from the FDA and a few billion are incomparable; however, reducing the FDA budget by 50% and reducing the military budget by 50% might be approximately equally radical suggestions.
So maybe we should be talking about log-budgets instead. Is there any example where such a calculation would produce counter-intuitive results?
I think you misunderstand, though. Just label the three budget options A, B, and C. There’s no need to rank them. There’s no need even to know what the question was about.
If you have a long questionnaire and many respondents, you have a big matrix, each column being a person’s responses to all the questions. These are labeled arbitrarily. Zero or one or something. Looking at the covariance matrix tells you which answers are correlated with which other answers; clusters and outliers appear. “Radicalness” is defined simply as low correlation with other columns: low likelihood of giving the same multiple-choice answers as other people did.
There’s arbitrariness in the choice of questions, but it’s not quite as arbitrary as you think.
I understand the general approach; my response was to this concrete budget-based proposal. However, you say:
There’s arbitrariness in the choice of questions, but it’s not quite as arbitrary as you think.
It is in fact extremely arbitrary. You’re avoiding the arbitrariness necessary for defining a normed vector space of political positions by offloading the same arbitrariness to the choice of questions in this model. Both approaches would likely correctly identify extreme outliers that are far from the mainstream, but the measured amount of variation within clustered groups would be, in both cases, an equally arbitrary reflection of the model author’s preconceptions.
To take an extreme example, you could have a hundred questions, ninety of which deal with various intricacies of Christian theology, while the rest deal with human universals. If you administered them to a representative sample of the world’s population, your model would tell you that the opinions of Christians span a much wider range of views than the opinions of non-Christians. The same principle applies—perhaps less obviously, but no less powerfully—to any other conceivable questionnaire.
This comment (I think) makes your point clearer to me. The problem isn’t so much in the “send your directed graph to a metric space” as it is “choose a basis of your metric space.”
I think there is a sense in which you could find “less arbitrary” choices, but I don’t think an algorithm exists among humans to find one reliably so I think you’re right.
I wouldn’t go so far as to say that you’ll produce arbitrary numbers no matter what. But I think you’re right that the method of measurement and analysis can make a huge difference to the observed result—it’s a volatile combination of statistics and qualitative politics. However, wasn’t part of the point of the post that this model is insufficient in the first place?
It’s all arbitrary here to some degree—it’s just messing around with the numbers. (Incidentally I wasn’t trying to imply that “political diversity” was a virtue.)
If you pick the axes to mean something, then they’ll be defined “to taste,” of course. If you don’t bother about giving them English-word labels, just let them be the first two components in PCA or something.
Honestly, I’m just messing around here. For crying out loud, it’s not meant to be Truth.
SarahC:
Trouble is, any concrete definition of this distance will be arbitrary. Even with a single dimension, it is only possible to construct an ordinal scale, and there is no objective way to define distance. With multiple dimensions, there is the additional problem of how to make different dimensions commensurable. By varying these arbitrary parameters, any such calculation can be manipulated to reach very different conclusions.
An assertion that people in one group have a wider range of views than those in another can be defended only insofar as common-sense clearly leads to such a conclusion. If this is not the case, then the answer given by all this flashy math will be determined by your own preconceptions you put into the model, not some objective truth.
If we used some kind of resource distribution to acquire the information; i.e. “you have a budget how much do you spend on what” you would (in addition to learning how little people know about economics) have some pretty solid numbers to work with in defining a metric.
The issues of budget allocation are only a subset of people’s political opinions. But even with concrete budget numbers, the metric still cannot be objective.
Suppose you have three people who advocate, respectively, a $700B military budget, a $350B military budget, and a $0 budget (i.e. complete abolition of the armed forces). Clearly, common sense tells us that the first two people have a large difference of opinion, but it pales in comparison with the extremism of the third one’s position, and it makes no sense to claim that the distances 1-2 and 2-3 are equal, even though the difference in numbers is the same. So again you have to introduce some arbitrary scaling to define this distance.
Moreover, the differences of opinion on different budget items cannot be compared directly based on just the amounts of money allocated. The budged of the FDA is only a few billion, but abolishing the FDA is a far more radical proposition that cutting back, say, the military budget by ten times what the FDA now gets. Again, you have to introduce some arbitrary criteria to compare these numbers.
So, ultimately, you’ll end up producing arbitrary numbers no matter what.
Both of your examples are ultimately arguments in favor of reasoning about ratios of budgets: going from $350B to $700B is a 100% increase, while going from $0 to $350B isn’t even defined. Perhaps $175B and $350B would be at a similar distance from each other.
Similarly, taking away a few billion from the FDA and a few billion are incomparable; however, reducing the FDA budget by 50% and reducing the military budget by 50% might be approximately equally radical suggestions.
So maybe we should be talking about log-budgets instead. Is there any example where such a calculation would produce counter-intuitive results?
I think you misunderstand, though. Just label the three budget options A, B, and C. There’s no need to rank them. There’s no need even to know what the question was about.
If you have a long questionnaire and many respondents, you have a big matrix, each column being a person’s responses to all the questions. These are labeled arbitrarily. Zero or one or something. Looking at the covariance matrix tells you which answers are correlated with which other answers; clusters and outliers appear. “Radicalness” is defined simply as low correlation with other columns: low likelihood of giving the same multiple-choice answers as other people did.
There’s arbitrariness in the choice of questions, but it’s not quite as arbitrary as you think.
I understand the general approach; my response was to this concrete budget-based proposal. However, you say:
It is in fact extremely arbitrary. You’re avoiding the arbitrariness necessary for defining a normed vector space of political positions by offloading the same arbitrariness to the choice of questions in this model. Both approaches would likely correctly identify extreme outliers that are far from the mainstream, but the measured amount of variation within clustered groups would be, in both cases, an equally arbitrary reflection of the model author’s preconceptions.
To take an extreme example, you could have a hundred questions, ninety of which deal with various intricacies of Christian theology, while the rest deal with human universals. If you administered them to a representative sample of the world’s population, your model would tell you that the opinions of Christians span a much wider range of views than the opinions of non-Christians. The same principle applies—perhaps less obviously, but no less powerfully—to any other conceivable questionnaire.
This comment (I think) makes your point clearer to me. The problem isn’t so much in the “send your directed graph to a metric space” as it is “choose a basis of your metric space.”
I think there is a sense in which you could find “less arbitrary” choices, but I don’t think an algorithm exists among humans to find one reliably so I think you’re right.
That’s certainly true. The choice of questions is entirely subjective.
I wouldn’t go so far as to say that you’ll produce arbitrary numbers no matter what. But I think you’re right that the method of measurement and analysis can make a huge difference to the observed result—it’s a volatile combination of statistics and qualitative politics. However, wasn’t part of the point of the post that this model is insufficient in the first place?
It’s all arbitrary here to some degree—it’s just messing around with the numbers. (Incidentally I wasn’t trying to imply that “political diversity” was a virtue.)
If you pick the axes to mean something, then they’ll be defined “to taste,” of course. If you don’t bother about giving them English-word labels, just let them be the first two components in PCA or something.
Honestly, I’m just messing around here. For crying out loud, it’s not meant to be Truth.