gwern comments on 2012 Survey Results

[gwern]

The average user matters a lot, I think… But since you insist, here’s the top 10% of each category:

R> sort(hpmor$KarmaScore, decreasing=T)[1:25]
 [1] 9122 6815 4887 4500 2782 2600 2545 2117 2000 1800 1300 1017 1000 1000  858  771  694  575  560
[20]  443  425  422  350  285  274
R> sort(other$KarmaScore, decreasing=T)[1:83]
 [1] 47384 32394 27418 15000 12200 11094 11000 10000  9000  8799  8000  8000  8000  6164  5000  5000
[17]  5000  5000  4658  4000  4000  4000  3960  3800  3693  3600  3500  3500  3500  3353  3300  3000
[33]  3000  3000  3000  3000  3000  3000  3000  2700  2500  2486  2400  2300  2204  2200  2100  2000
[49]  2000  2000  2000  2000  1977  1975  1900  1800  1800  1800  1750  1700  1653  1650  1648  1600
[65]  1590  1540  1520  1500  1500  1500  1500  1500  1500  1400  1253  1250  1200  1200  1115  1095
[81]  1044  1000  1000

The top MoR referral user is somewhere around 10th place in the other group (which is 3.3x larger).

[Qiaochu_Yuan]

The average user that sticks around might matter a lot, but people with low karma are probably less likely to stick around so they’ll have less of an impact (positive or negative) on the community. So maybe look at the distribution of karma, but among veteran users resp. veteran MoR users?

[gwern]

What’s ‘veteran’? (And how many ways do you want to slice the data anyway...)

[Vaniver]

I imagine that when you divide karma by months in the community (while still restricting yourself to the top ten percent of absolute karma) the MoR contributors will look better. I’ll do it tonight if you don’t.

[gwern]

They do a bit better at the top; the sample size at “top 10%” is getting small enough that tests are losing power, though:

R> lw <- read.csv("lw-survey/2012.csv")
R>
R> hpmor <- lw[as.character(lw$Referrals) == "Referred by Harry Potter and the Methods of Rationality",]
R> other <- lw[as.character(lw$Referrals) != "Referred by Harry Potter and the Methods of Rationality",]
R>
R> hpmor <- hpmor[order(hpmor$KarmaScore, decreasing=TRUE),][1:25,]
R> other <- other[order(other$KarmaScore, decreasing=TRUE),][1:83,]
R>
R> hpmortime <- hpmor$KarmaScore / as.numeric(as.character(hpmor$TimeinCommunity))
R> hpmortime <- hpmortime[!is.na(hpmortime) & !is.nan(hpmortime) & !is.infinite(hpmortime) ]
R> othertime <- other$KarmaScore / as.numeric(as.character(other$TimeinCommunity))
R> othertime <- othertime[!is.na(othertime) & !is.nan(othertime) & !is.infinite(othertime) ]
R>
R> sort(hpmortime, decreasing=TRUE)
 [1] 506.78 300.00 283.96 203.62 138.46 133.95 117.61 115.92  72.22  66.67  59.09  50.00  36.92
[14]  35.05  35.00  33.90  28.60  26.67  24.82  23.96  20.36  19.28  17.71  11.91
R> sort(othertime, decreasing=TRUE)
 [1] 1895.36  647.88  456.97  338.89  263.16  250.00  250.00  235.71  184.90  183.33  173.91  166.67
[13]  165.00  146.93  146.65  145.83  142.86  133.33  133.33  133.33  125.00  125.00  125.00  116.45
[25]  102.73  100.00   97.22   84.38   83.33   83.33   83.33   83.33   75.00   75.00   74.51   72.00
[37]   69.60   68.88   66.67   66.67   63.33   61.11   60.71   60.34   58.33   57.14   55.95   53.43
[49]   52.17   50.00   50.00   50.00   50.00   48.48   46.46   44.12   43.75   41.67   41.43   40.00
[61]   39.66   36.36   35.91   33.33   33.33   31.67   31.32   30.00   30.00   30.00   27.50   27.47
[73]   26.95   25.33   25.00   25.00   24.06   23.33   22.73   18.25   16.67   16.67
R>
R> t.test(hpmortime,othertime)

    Welch Two Sample t-test

data:  hpmortime and othertime
t = -0.544, df = 72.4, p-value = 0.5881
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -87.52  49.99
sample estimates:
mean of x mean of y
    98.44    117.20