calculated performance rating for bullet games for each day
calculated anki accuracy (as measured by (1 - again%)) for each day
adjusted performance rating according to time by fitting an OLS model to predict perf rating with days since beginning of the dataset, then subtracting the model’s prediction (should yield a normal distribution—this model had an R^2 of 0.4)
fit an OLS model to predict anki accuracy given the adjusted performance rating
This has an R^2 of 0.016, and the coefficient is ~5.5e-05 (though it is pretty significant). So a performance rating of 1000 higher than predicted only yields a boost of ~5% additional accuracy on anki. Since the adjusted performance rating has a standard deviation of 208 points, that means if you’re having a “top cognition” day that’s 2 std’s above average, that’s only 2% higher anki accuracy. Not a lot.
Of note: using a “locally smoothed” performance rating (where I smoothed the perf rating, then subtracted that from the perf rating to get a residual) yielded no significant correlation between anki accuracy and perf rating. Arguably this is a stronger bit of evidence—the above (naïvely) assumes that the perf rating goes up linearly with time, but this version is able to deal with plateaus and different slopes in increasing/decreasing rating.
I’m open for code/analysis review if anyone wants to double check my work.
Here is what I did (n=524):
calculated performance rating for bullet games for each day
calculated anki accuracy (as measured by (1 - again%)) for each day
adjusted performance rating according to time by fitting an OLS model to predict perf rating with days since beginning of the dataset, then subtracting the model’s prediction (should yield a normal distribution—this model had an R^2 of 0.4)
fit an OLS model to predict anki accuracy given the adjusted performance rating
This has an R^2 of 0.016, and the coefficient is ~5.5e-05 (though it is pretty significant). So a performance rating of 1000 higher than predicted only yields a boost of ~5% additional accuracy on anki. Since the adjusted performance rating has a standard deviation of 208 points, that means if you’re having a “top cognition” day that’s 2 std’s above average, that’s only 2% higher anki accuracy. Not a lot.
Of note: using a “locally smoothed” performance rating (where I smoothed the perf rating, then subtracted that from the perf rating to get a residual) yielded no significant correlation between anki accuracy and perf rating. Arguably this is a stronger bit of evidence—the above (naïvely) assumes that the perf rating goes up linearly with time, but this version is able to deal with plateaus and different slopes in increasing/decreasing rating.
I’m open for code/analysis review if anyone wants to double check my work.