On social media lots of people are basically saying that it’s so obvious that Trump is a racist that everyone who voted for Trump knew they were voting for a racist. There could be value to a model where nature first decides how racist a candidate is, then two people get different signals of how racist that candidate is, then finally you calculate how strong your signal has to be for you to be confident that the other person should be confident that the candidate is a racist.
I’m assuming that the reason these people are pushing the idea that at least 50% of the US population are racist is because they want to normalise racism. Otherwise, they’rd be shooting themselves in the foot, normalising the very idea they are trying to stop, and no-one could be that stupid … right?
Huh? That doesn’t make much sense. Let’s translate it from the political minefield to standard statistics.
You have some value (say, the population mean) which you don’t know, but which exists. You get an estimate of it (say, the sample mean) and another guy gets another estimate (say, the mean of a different sample). You are asking what should be the statistical significance of your estimate in order for you to be confident that the other guy must believe your estimate and not his own.
You have some value (say, the population mean) which you don’t know, but which exists. You get an estimate of it (say, the sample mean) and another guy gets another estimate (say, the mean of a different sample). You are asking what should be the statistical significance of your estimate in order for you to be confident that the other guy is with high probability confident that based on his estimate the population mean is above some threshold.
Less abstractly. Nature picks a number between 0 and 1. I get an estimate of this number, and the other guy gets a separate estimate. We don’t see each other’s estimate but we know how it was picked. For what values of my estimate can I be more than 90% confident that the other guy is more than 90% confident that nature picked a number above, say, .8?
So let’s say there is a value which we can’t observe directly. Our indirect observations come with noise. To keep things simple let’s assume that our observations are iid and that the noise is zero-mean (unbiased) additive Gaussian. Let’s also assume that the noise is the same for you and for the other observer (if you don’t know how noisy the other guy’s observations are, you won’t be able to answer the question).
Let’s say you have n observations. Let’s call the standard deviation of the noise ‘sigma’ so that noise ~N(0, sigma). You care about “higher” so the significance is going to be one-tailed. You want 90% confidence and our noise is Gaussian, so in standard errors (SE) we want to be about 1.3 standard errors higher than the threshold.
The SE is just sigma / sqrt(n). This means that your estimate has to be greater than (0.8 + 1.3 * sigma / sqrt(n)) for you to have 90% confidence the true value is larger than 0.8.
But your question is different. You want 90% confidence not that the true value is >0.8, but that 90% of the samples will show 90% confidence that the value is >0.8. That’s not hard.
An estimate will provide 90% confidence if it is greater than 0.8 + 1.3 sigma / sqrt(n). The estimates’ standard deviation is sigma/sqrt(n), so we just repeat: you can “be more than 90% confident that the other guy is more than 90% confident” if your estimate is above (0.8 + 1.3 sigma / sqrt(n)) + (1.3 sigma / sqrt(n)) = 0.8 + 2.6 sigma / sqrt(n).
So if you see an estimate that’s more than 2.6 SEs greater than 0.8 (which would lead to your own confidence about the true value being in excess of 99%), you can be 90% sure that the other guy is 90% sure.
Does it make sense to do the “this is what the world would look like if our sampling methodologies were the same” calculation when you have strong reasons to suspect that the sampling methodologies are not the same?
The calculation is more to convince members of the rationalist community that you need extremely strong evidence to believe that rationalist Trump voters thought Trump was racist. Assuming differing sampling methodologies would strengthen the result beyond the 2.6 SEs that Lumifer calculated.
On social media lots of people are basically saying that it’s so obvious that Trump is a racist that everyone who voted for Trump knew they were voting for a racist. There could be value to a model where nature first decides how racist a candidate is, then two people get different signals of how racist that candidate is, then finally you calculate how strong your signal has to be for you to be confident that the other person should be confident that the candidate is a racist.
I’m assuming that the reason these people are pushing the idea that at least 50% of the US population are racist is because they want to normalise racism. Otherwise, they’rd be shooting themselves in the foot, normalising the very idea they are trying to stop, and no-one could be that stupid … right?
\s
Huh? That doesn’t make much sense. Let’s translate it from the political minefield to standard statistics.
You have some value (say, the population mean) which you don’t know, but which exists. You get an estimate of it (say, the sample mean) and another guy gets another estimate (say, the mean of a different sample). You are asking what should be the statistical significance of your estimate in order for you to be confident that the other guy must believe your estimate and not his own.
That’s… a weird question.
My translation:
You have some value (say, the population mean) which you don’t know, but which exists. You get an estimate of it (say, the sample mean) and another guy gets another estimate (say, the mean of a different sample). You are asking what should be the statistical significance of your estimate in order for you to be confident that the other guy is with high probability confident that based on his estimate the population mean is above some threshold.
Less abstractly. Nature picks a number between 0 and 1. I get an estimate of this number, and the other guy gets a separate estimate. We don’t see each other’s estimate but we know how it was picked. For what values of my estimate can I be more than 90% confident that the other guy is more than 90% confident that nature picked a number above, say, .8?
That’s a fairly straightforward question.
So let’s say there is a value which we can’t observe directly. Our indirect observations come with noise. To keep things simple let’s assume that our observations are iid and that the noise is zero-mean (unbiased) additive Gaussian. Let’s also assume that the noise is the same for you and for the other observer (if you don’t know how noisy the other guy’s observations are, you won’t be able to answer the question).
Let’s say you have n observations. Let’s call the standard deviation of the noise ‘sigma’ so that noise ~N(0, sigma). You care about “higher” so the significance is going to be one-tailed. You want 90% confidence and our noise is Gaussian, so in standard errors (SE) we want to be about 1.3 standard errors higher than the threshold.
The SE is just sigma / sqrt(n). This means that your estimate has to be greater than (0.8 + 1.3 * sigma / sqrt(n)) for you to have 90% confidence the true value is larger than 0.8.
But your question is different. You want 90% confidence not that the true value is >0.8, but that 90% of the samples will show 90% confidence that the value is >0.8. That’s not hard.
An estimate will provide 90% confidence if it is greater than 0.8 + 1.3 sigma / sqrt(n). The estimates’ standard deviation is sigma/sqrt(n), so we just repeat: you can “be more than 90% confident that the other guy is more than 90% confident” if your estimate is above (0.8 + 1.3 sigma / sqrt(n)) + (1.3 sigma / sqrt(n)) = 0.8 + 2.6 sigma / sqrt(n).
So if you see an estimate that’s more than 2.6 SEs greater than 0.8 (which would lead to your own confidence about the true value being in excess of 99%), you can be 90% sure that the other guy is 90% sure.
Thanks!
Does it make sense to do the “this is what the world would look like if our sampling methodologies were the same” calculation when you have strong reasons to suspect that the sampling methodologies are not the same?
The calculation is more to convince members of the rationalist community that you need extremely strong evidence to believe that rationalist Trump voters thought Trump was racist. Assuming differing sampling methodologies would strengthen the result beyond the 2.6 SEs that Lumifer calculated.