What sort of substantiation were you looking for? The article pretty clearly states that the claims about the effects of the camp were based on exit surveys, and that the impact of the camp is demonstrated by the projects the camp grads are now working on. You could debate whether those are good measures, but we don’t exactly have better ones.
I agree it would be nice if we could come up with standardized tests for rationality and then test whether camp attendance improves scores, but even if this were possible it hardly seems the best conceivable use of SIAIs resources.
What sort of substantiation were you looking for? The article pretty clearly states that the claims about the effects of the camp were based on exit surveys, and that the impact of the camp is demonstrated by the projects the camp grads are now working on.
Well, I’d want to know what thes supposed projects are, what progress they’ve made, how well people are sticking to them, etc. It’s not much help to mention self-assessed exit surveys, given that people are remarkably poor judges of their own skill. The self-assessments would really only measure how much fun the participants had, which was not the standard it claimed to judge itself by.
They could also post improvements on tests—since there were specific skills taught, surely there’s an easy before/after they can come up with. For the males (given what I remember about what lukeprog was going to teach), they can report (anonymized) “dating effectiveness improvement”.
For a smashing success, there’s remarkably little I can see to distinguish it from non-success. (And when such evidence does come, it hope I’m not criticized for asking for it.)
I agree the post is a bit over the top. From what I saw, At the end of the minicamp, everyone, including myself, felt really good about how it had gone. The attendees all felt that they had gotten really valuable lessons out of it. But of course that feeling so short after learning the material isn’t that strong of evidence. I agree better measurement would be a really good thing, though I’m not certain how you would go about that.
Here are some things I currently think I got out of the minicamp and some reasons why:
The realization that having a more accurate model of my mind could be quite valuable and that science had more useful things to say about my brain than just the heuristics and biases literature. In particular, I’m thinking of the research on how (some) emotions work (affective neuroscience) and the research on motivation. Luke and Academian were fairly convincing that their knowledge of these fields allowed them to be more successful. I also don’t think I had been taking the idea that my brain is composed of many subprocesses many of which influence my behavior but are mostly opaque to my conscious mind seriously enough for practical purposes.
I realized that being in close contact with smart, relatively rational people trying to optimize themselves makes optimizing yourself quite a bit easier. It’s much easier to approach people for ideas on how to optimize and the quality of ideas are better. It’s also more fun! This has made me more excited about the Seattle LW group.
I realized that being fashionable is more important than I thought (I just hadn’t given it very much thought before). I also got very actionable information on how to improve my fashion (general rules, ways to learn more, how to go about buying stuff, etc.). I also realized I was signaling stuff I didn’t want to signal, so I’m changing my general look. Basically, luke went through a list of ‘don’ts’ and my typical style was explicitly mentioned.
I figured out I should be putting more energy into my work and less on other things. I’ve started collecting data on my productivity and trying to train myself to notice when I’m getting distracted and to do something about it.
Anna had a lot of useful seeming tricks for getting yourself to engage in actual thinking when you need it to (recognizing characteristic thinking substitutes, noticing various kinds of warning signs, etc.). I’ve used a at least two of these to change my mind on something at work.
The mere act of writing down important goals and problems and asking obvious seeming questions can generate useful insights. Further discussion with reasonable people generates even more insights. This happened to me and others several times. The Seattle LW group has done this once already and it was pretty useful, so we’re going to focus more on it in the future.
I realized I hadn’t paid nearly enough attention to the notion of an Ugh Field. I noticed I have been (and probably continue to be) heavily affected by something along these lines. This concept has helped me get a lot of stuff done that I otherwise wouldn’t have.
There’s more, but I’ll leave it there for now and maybe come back to it.
One thing that is being doing, that does provide data is that people are submitting before and after fashion photos a month after the camp (the aim is to get people to follow through on implementing fashion changes). Obviously this isn’t perfect, but it does provide some data about how effective that part of the camp has been.
As for me, I evaluated my life immediately after mini-camp and made some changes resulting in (conservatively estimated) an extra 7 quality-adjusted-hours per week and $5000 per year. I could have but did not make those changes without the skills/impetus/thinking-framework I gained from mini-camp (possibly any “improve!” week-long retreat would have had a similar effect). I am in the process of making more personal improvement changes whose benefits are harder to estimate, such as but not limited to improving my look, training the ability to instill chosen wants as visceral desires in myself, becoming involved in a community of cognomancers, and twice-weekly training of epistemic rationality subskills like calibration and noticing what my beliefs prohibit.
I predict that surface level changes will be visible now and deeper changes won’t show up almost at all on any easy metrics until after 2-3 months and after a year I will have changed quite a bit for the better, as measured by asking people who know me about the difference between me and me a year ago compared to the difference between me a year ago and me two years ago.
Most of the learning I did in the mini-camp looks like it requires training before I get most of the benefit from it; I am currently training. I will go over the wiki page on the Problem of Verifying Rationality on Saturday to see if we already have tests or ideas for tests that make sense to take now and then 2 months from now. Others’ timelines may differ.
That internet website’s name infringes on the LessWrong trademark. That is bad. Humans should be more respectful of intellectual property rights, so that Pareto-efficient ideas will be discovered and instantiated.
I suppose that if nobody confuses your site with official LessWrong internet webstes, there should be no problem.
By my last sentence, I just mean that you need respect for intellectual property in order to get the Pareto-optimal level of investment in ideas, and so you should be respectful of such claims.
upvoted, for providing specific constructive suggests for SIAI. I hope no one will meet requests such as these with derision, and I’m not entirely sure why you’d expect that.
Thanks. I’ve toned down the comment a bit. And I say this because some of the people involved have responded foot-draggingly and rudely to requests like this that I’ve made.
It’s good to continue to ask for that kind of thing, but I think sounding adversarial probably makes you less likely to get good responses not more. Better to sound like the friend who relentlessly dogs you to be better at giving responses. I understand the temptation, though, because I get similar temptations.
Since I attended the minicamp, I’ll respond later tonight.
Nice PR. Any substantiation for any of the claims in the link? (Apologies if I’m not supposed to ask that.)
What sort of substantiation were you looking for? The article pretty clearly states that the claims about the effects of the camp were based on exit surveys, and that the impact of the camp is demonstrated by the projects the camp grads are now working on. You could debate whether those are good measures, but we don’t exactly have better ones.
I agree it would be nice if we could come up with standardized tests for rationality and then test whether camp attendance improves scores, but even if this were possible it hardly seems the best conceivable use of SIAIs resources.
Well, I’d want to know what thes supposed projects are, what progress they’ve made, how well people are sticking to them, etc. It’s not much help to mention self-assessed exit surveys, given that people are remarkably poor judges of their own skill. The self-assessments would really only measure how much fun the participants had, which was not the standard it claimed to judge itself by.
They could also post improvements on tests—since there were specific skills taught, surely there’s an easy before/after they can come up with. For the males (given what I remember about what lukeprog was going to teach), they can report (anonymized) “dating effectiveness improvement”.
For a smashing success, there’s remarkably little I can see to distinguish it from non-success. (And when such evidence does come, it hope I’m not criticized for asking for it.)
I agree the post is a bit over the top. From what I saw, At the end of the minicamp, everyone, including myself, felt really good about how it had gone. The attendees all felt that they had gotten really valuable lessons out of it. But of course that feeling so short after learning the material isn’t that strong of evidence. I agree better measurement would be a really good thing, though I’m not certain how you would go about that.
Here are some things I currently think I got out of the minicamp and some reasons why:
The realization that having a more accurate model of my mind could be quite valuable and that science had more useful things to say about my brain than just the heuristics and biases literature. In particular, I’m thinking of the research on how (some) emotions work (affective neuroscience) and the research on motivation. Luke and Academian were fairly convincing that their knowledge of these fields allowed them to be more successful. I also don’t think I had been taking the idea that my brain is composed of many subprocesses many of which influence my behavior but are mostly opaque to my conscious mind seriously enough for practical purposes.
I realized that being in close contact with smart, relatively rational people trying to optimize themselves makes optimizing yourself quite a bit easier. It’s much easier to approach people for ideas on how to optimize and the quality of ideas are better. It’s also more fun! This has made me more excited about the Seattle LW group.
I realized that being fashionable is more important than I thought (I just hadn’t given it very much thought before). I also got very actionable information on how to improve my fashion (general rules, ways to learn more, how to go about buying stuff, etc.). I also realized I was signaling stuff I didn’t want to signal, so I’m changing my general look. Basically, luke went through a list of ‘don’ts’ and my typical style was explicitly mentioned.
I figured out I should be putting more energy into my work and less on other things. I’ve started collecting data on my productivity and trying to train myself to notice when I’m getting distracted and to do something about it.
Anna had a lot of useful seeming tricks for getting yourself to engage in actual thinking when you need it to (recognizing characteristic thinking substitutes, noticing various kinds of warning signs, etc.). I’ve used a at least two of these to change my mind on something at work.
The mere act of writing down important goals and problems and asking obvious seeming questions can generate useful insights. Further discussion with reasonable people generates even more insights. This happened to me and others several times. The Seattle LW group has done this once already and it was pretty useful, so we’re going to focus more on it in the future.
I realized I hadn’t paid nearly enough attention to the notion of an Ugh Field. I noticed I have been (and probably continue to be) heavily affected by something along these lines. This concept has helped me get a lot of stuff done that I otherwise wouldn’t have.
There’s more, but I’ll leave it there for now and maybe come back to it.
One thing that is being doing, that does provide data is that people are submitting before and after fashion photos a month after the camp (the aim is to get people to follow through on implementing fashion changes). Obviously this isn’t perfect, but it does provide some data about how effective that part of the camp has been.
As for me, I evaluated my life immediately after mini-camp and made some changes resulting in (conservatively estimated) an extra 7 quality-adjusted-hours per week and $5000 per year. I could have but did not make those changes without the skills/impetus/thinking-framework I gained from mini-camp (possibly any “improve!” week-long retreat would have had a similar effect). I am in the process of making more personal improvement changes whose benefits are harder to estimate, such as but not limited to improving my look, training the ability to instill chosen wants as visceral desires in myself, becoming involved in a community of cognomancers, and twice-weekly training of epistemic rationality subskills like calibration and noticing what my beliefs prohibit.
I predict that surface level changes will be visible now and deeper changes won’t show up almost at all on any easy metrics until after 2-3 months and after a year I will have changed quite a bit for the better, as measured by asking people who know me about the difference between me and me a year ago compared to the difference between me a year ago and me two years ago.
Most of the learning I did in the mini-camp looks like it requires training before I get most of the benefit from it; I am currently training. I will go over the wiki page on the Problem of Verifying Rationality on Saturday to see if we already have tests or ideas for tests that make sense to take now and then 2 months from now. Others’ timelines may differ.
Guy’s blog eachdaylesswrong.blogspot.com/ may also provide more evidence (let me know if you don’t want the link here).
That internet website’s name infringes on the LessWrong trademark. That is bad. Humans should be more respectful of intellectual property rights, so that Pareto-efficient ideas will be discovered and instantiated.
I did not think of that. Is it a problem worth fixing? I’m not sure what you mean by your last sentence.
I suppose that if nobody confuses your site with official LessWrong internet webstes, there should be no problem.
By my last sentence, I just mean that you need respect for intellectual property in order to get the Pareto-optimal level of investment in ideas, and so you should be respectful of such claims.
upvoted, for providing specific constructive suggests for SIAI. I hope no one will meet requests such as these with derision, and I’m not entirely sure why you’d expect that.
Thanks. I’ve toned down the comment a bit. And I say this because some of the people involved have responded foot-draggingly and rudely to requests like this that I’ve made.
It’s good to continue to ask for that kind of thing, but I think sounding adversarial probably makes you less likely to get good responses not more. Better to sound like the friend who relentlessly dogs you to be better at giving responses. I understand the temptation, though, because I get similar temptations.
Since I attended the minicamp, I’ll respond later tonight.
Seeing the actual results and answers on the survey would be a good start.
We’re all such attractive people!
Our one advantage over the Catholic Church.
Yea make me even more jealous i wasn’t chosen to go :P