How was the control group selected? Did you select a pool of candidates larger than you could accept then randomly take a subset of these as a control? If not then calling it a ‘control group’ is borderline at best.
The prior expectation of the influence of one week of training on personal success over a year is far lower than that of various personal and environmental qualities in the individual. This being the case it is more reasonable to attribute differences in progress between the groups to the higher potential for growth in the chosen minicampers. This primarily reflects well on the ability of the Singinst rationality trainers to identify indicators of future success—a rather important skill in its own right!
A good point. The control group was of folks who made it through the initial screening but not the final screening, so, yes, there are differences. We explicitly discussed the possibility of randomizing admissions, but, for our first go, elected to admit the 25 people we most wanted, and to try randomizing some future events if the first worked well enough to warrant follow-ups (which it did). It is a bit of a hit to the data-gathering, but it wasn’t growth potential as such that we were selecting for—for example, younger applicants were less likely to have cool accomplishments and therefore less likely to get in, although they probably have more growth potential—so there should still be evidence in the results.
Also, we marked down which not-admitted applicants were closest to the cut-off line (there were a number of close calls; I really wished we had more than 25 spaces), so we can gain a bit of data by seeing if they were similar to the minicamp group or to the rest of the controls.
How was the control group selected? Did you select a pool of candidates larger than you could accept then randomly take a subset of these as a control? If not then calling it a ‘control group’ is borderline at best.
The prior expectation of the influence of one week of training on personal success over a year is far lower than that of various personal and environmental qualities in the individual. This being the case it is more reasonable to attribute differences in progress between the groups to the higher potential for growth in the chosen minicampers. This primarily reflects well on the ability of the Singinst rationality trainers to identify indicators of future success—a rather important skill in its own right!
A good point. The control group was of folks who made it through the initial screening but not the final screening, so, yes, there are differences. We explicitly discussed the possibility of randomizing admissions, but, for our first go, elected to admit the 25 people we most wanted, and to try randomizing some future events if the first worked well enough to warrant follow-ups (which it did). It is a bit of a hit to the data-gathering, but it wasn’t growth potential as such that we were selecting for—for example, younger applicants were less likely to have cool accomplishments and therefore less likely to get in, although they probably have more growth potential—so there should still be evidence in the results.
Also, we marked down which not-admitted applicants were closest to the cut-off line (there were a number of close calls; I really wished we had more than 25 spaces), so we can gain a bit of data by seeing if they were similar to the minicamp group or to the rest of the controls.