Here.
VincentYu
Karma: 2,203
I suggest including the Big Five Inventory (BFI) in the survey itself. I’ve created an example of this on Google Forms. (I’ve reordered the inventory such that the first 11 items constitute the BFI-10, so that respondents can choose between the 44-item and 11-item versions).
The BFI is the inventory that was used in the online test to which the 2012 LW census linked. See also my comment about this in the 2012 LW census thread.
Unfortunately, my university library reports that they have exhausted all possible sources and no library was able to supply this paper.
First, let me point out that the “behavioral changes” that the authors described were investigated over only three posts subsequent to each positive/negative evaluation, so it is unclear whether these effects remain over the long term.
Second, I find questionable the authors’ conclusion that negative evaluations cause the subsequent decline in post quality and increase in post frequency, since they did not control the positive/negative evaluations. They model the positive/negative evaluations as random acts of chance (which is what we want for an RCT) and justify this by reporting that their bigram classifier assigns no difference in quality between the positively- and negatively-evaluated posts (across two posts by a pair of matched subjects). However, I find it likely that their classifier makes sufficiently many misclassifications to call into question their conclusion.
For instance, if bad posts have a tendency to occur in streaks of frequent posts (as is the case in flame wars#Flame_war)), then we can explain their observations without assigning causal potency to negative evaluations: once in a while the classifier will erroneously assign a high quality to a bad post near the start of a flame war, but on average it will correctly assign low qualities to the subsequent three posts by the same poster in the flame war, and thus we see the effects that the authors described (without assigning any causal effect to the negative evaluation given by other users to the post near the start of the flame war). To test this explanation, the authors can ask the Crowdflower workers (p. 4) to label each b_0 (described on p. 5) to check whether their classifier is indeed misclassifying b_0 by assigning it too high a quality.
Since the authors did not conduct an RCT, we can come up with many alternative explanations, and I find them plausible. (Is it feasible to conduct an RCT on a site featuring upvotes and downvotes? Yes, it’s been done before.)
Despite my criticisms, I think the paper is not bad. I just don’t think the authors’ methods provide sufficient evidence to warrant their seemingly strong confidence in their conclusions.
Here.
Requested.
Requested.
Here.
IQ is normally distributed because the genetics is a lot of small additive variables.
IQ is normally distributed because the distribution of raw test scores is standardized to a normal distribution.
Here.
The article to which this letter is responding to is Stanovich and West (2014).
Requested.
Here.
Requested.
A “holy war” between Bayesians and frequentists exists in the modern academic literature for statistics, machine learning, econometrics, and philosophy (this is a non-exhaustive list).
Bradley Efron, who is arguably the most accomplished statistician alive, wrote the following in a commentary for Science in 2013 [1]:
The term “controversial theorem” sounds like an oxymoron, but Bayes’ theorem has played this part for two-and-a-half centuries. Twice it has soared to scientific celebrity, twice it has crashed, and it is currently enjoying another boom. The theorem itself is a landmark of logical reasoning and the first serious triumph of statistical inference, yet is still treated with suspicion by most statisticians. There are reasons to believe in the staying power of its current popularity, but also some signs of trouble ahead.
[...]
Bayes’ 1763 paper was an impeccable exercise in probability theory. The trouble and the subsequent busts came from overenthusiastic application of the theorem in the absence of genuine prior information, with Pierre-Simon Laplace as a prime violator. Suppose that in the twins example we lacked the prior knowledge that one-third of twins are identical. Laplace would have assumed a uniform distribution between zero and one for the unknown prior probability of identical twins, yielding 2⁄3 rather than 1⁄2 as the answer to the physicists’ question. In modern parlance, Laplace would be trying to assign an “uninformative prior” or “objective prior”, one having only neutral effects on the output of Bayes’ rule. Whether or not this can be done legitimately has fueled the 250-year controversy.
Frequentism, the dominant statistical paradigm over the past hundred years, rejects the use of uninformative priors, and in fact does away with prior distributions entirely. In place of past experience, frequentism considers future behavior. An optimal estimator is one that performs best in hypothetical repetitions of the current experiment. The resulting gain in scientific objectivity has carried the day, though at a price in the coherent integration of evidence from different sources, as in the FiveThirtyEight example.
The Bayesian-frequentist argument, unlike most philosophical disputes, has immediate practical consequences.
In another paper published in 2013, Efron wrote [2]:
The two-party system [Bayesian and frequentist] can be upsetting to statistical consumers, but it has been a good thing for statistical researchers — doubling employment, and spurring innovation within and between the parties. These days there is less distance between Bayesians and frequentists, especially with the rise of objective Bayesianism, and we may even be heading toward a coalition government.
The two philosophies, Bayesian and frequentist, are more orthogonal than antithetical. And of course, practicing statisticians are free to use whichever methods seem better for the problem at hand — which is just what I do.
Thirty years ago, Efron was more critical of Bayesian statistics [3]:
A summary of the major reasons why Fisherian and NPW [Neyman–Pearson–Wald] ideas have shouldered Bayesian theory aside in statistical practice is as follows:
Ease of use: Fisher’s theory in particular is well set up to yield answers on an easy and almost automatic basis.
Model building: Both Fisherian and NPW theory pay more attention to the preinferential aspects of statistics.
Division of labor: The NPW school in particular allows interesting parts of a complicated problem to be broken off and solved separately. These partial solutions often make use of aspects of the situation, for example, the sampling plan, which do not seem to help the Bayesian.
Objectivity: The high ground of scientific objectivity has been seized by the frequentists.
None of these points is insurmountable, and in fact, there have been some Bayesian efforts on all four. In my opinion a lot more such effort will be needed to fulfill Lindley’s prediction of a Bayesian 21st century.
The following bit of friendly banter in 1965 between M. S. Bartlett and John W. Pratt shows that the holy war was ongoing 50 years ago [4]:
Bartlett: I am not being altogether facetious in suggesting that, while non-Bayesians should make it clear in their writings whether they are non-Bayesian Orthodox or non-Bayesian Fisherian, Bayesians should also take care to distinguish their various denominations of Bayesian Epistemologists, Bayesian Orthodox and Bayesian Savages. (In fairness to Dr Good, I could alternatively have referred to Bayesian Goods; but, oddly enough, this did not sound so good.)
Pratt: Professor Bartlett is correct in classifying me a Bayesian Savage, though I might take exception to his word order. On the whole, I would rather be called a Savage Bayesian than a Bayesian Savage. Of course I can quite see that Professor Bartlett might not want to admit the possibility of a Good Bayesian.
For further reading I recommend [5], [6], [7].
[1]: Efron, Bradley. 2013. “Bayes’ Theorem in the 21st Century.” Science 340 (6137) (June 7): 1177–1178. doi:10.1126/science.1236536.
[2]: Efron, Bradley. 2013. “A 250-Year Argument: Belief, Behavior, and the Bootstrap.” Bulletin of the American Mathematical Society 50 (1) (April 25): 129–146. doi:10.1090/S0273-0979-2012-01374-5.
[3]: Efron, B. 1986. “Why Isn’t Everyone a Bayesian?” American Statistician 40 (1) (February): 1–11. doi:10.1080/00031305.1986.10475342.
[4]: Pratt, John W. 1965. “Bayesian Interpretation of Standard Inference Statements.” Journal of the Royal Statistical Society: Series B (Methodological) 27 (2): 169–203. http://www.jstor.org/stable/2984190.
[5]: Senn, Stephen. 2011. “You May Believe You Are a Bayesian but You Are Probably Wrong.” Rationality, Markets and Morals 2: 48–66. http://www.rmm-journal.com/htdocs/volume2.html.
[6]: Gelman, Andrew. 2011. “Induction and Deduction in Bayesian Data Analysis.” Rationality, Markets and Morals 2: 67–78. http://www.rmm-journal.com/htdocs/volume2.html.
[7]: Gelman, Andrew, and Christian P. Robert. 2012. “‘Not Only Defended but Also Applied’: The Perceived Absurdity of Bayesian Inference”. Statistics; Theory. arXiv (June 28).
I went through the literature on background music in September 2012; here is a dump of 38 paper references. Abstracts can be found by searching here and I can provide full texts on request.
Six papers that I starred in my reference manager (with links to full texts):
Chamorro-Premuzic, Tomas, Montserrat Gomà-i-Freixanet, Adrian Furnham, and Anna Muro. 2009. “Personality, Self-Estimated Intelligence, and Uses of Music: A Spanish Replication and Extension Using Structural Equation Modeling.” Psychology of Aesthetics, Creativity, and the Arts 3 (3): 149–155. doi:10.1037/a0015342.
Dobbs, Stacey, Adrian Furnham, and Alastair McClelland. 2011. “The Effect of Background Music and Noise on the Cognitive Test Performance of Introverts and Extraverts.” Applied Cognitive Psychology 25 (2) (March 23): 307–313. doi:10.1002/acp.1692.
Hallam, Susan. 2012. “The Effects of Background Music on Health and Wellbeing.” In Music, Health, and Wellbeing, edited by Raymond A. R. MacDonald, Gunter Kreutz, and Laura Mitchell. Oxford, UK: Oxford University Press. doi:10.1093/acprof:oso/9780199586974.003.0032.
Perham, Nick, and Joanne Vizard. 2011. “Can Preference for Background Music Mediate the Irrelevant Sound Effect?” Applied Cognitive Psychology 25 (4) (July 20): 625–631. doi:10.1002/acp.1731.
Schellenberg, E. Glenn. 2012. “Cognitive Performance after Listening to Music: A Review of the Mozart Effect.” In Music, Health, and Wellbeing, edited by Raymond A. R. MacDonald, Gunter Kreutz, and Laura Mitchell. Oxford, UK: Oxford University Press. doi:10.1093/acprof:oso/9780199586974.003.0022.
Waterhouse, Lynn. 2006. “Multiple Intelligences, the Mozart Effect, and Emotional Intelligence: A Critical Review.” Educational Psychologist 41 (4) (December): 207–225. doi:10.1207/s15326985ep4104_1.
One-word summary of the academic literature on the effects of listening to background music (as of September 2012): unclear
The post is “Knowing About Biases Can Hurt People”. See also the wiki page on fully general counterarguments.