In order to apply, you have to list 4 people who would definitely know how awesome you’re being a year from now, and give their contact info. Then, choose 1 of those people 6 months later and 1 person a year later and ask them how awesome the person is being. When you ask, include a “rubric” of various stories of various awesomeness levels, in which the highest levels are not always just $$$ but sometimes are. Ask the people you’re asking to please not contact the person specifically to check awesomeness, because that could introduce bias (“this person is checking, that makes me remember the workshop I did, and feel awesome”).
The 4 people should probably include no couples. Your family, long-term friends...
The one way this breaks down is facebook. I mean, if your interaction with each person is separate, and the workshop makes you seem more awesome to each of 4 people, it is working. But if it just makes you post more upbeat things on Facebook, that might not translate to actual awesomeness. But I think that’s a really minor factor.
Sure, it’s gonna be a noisy and imperfect measurement. You will have to look at standard deviations and calculate power (including burning all 4 contacts for some people to see the within-subject variance). Also, correct for demographic info on contacts, and various other tricks to increase power. But one way or another, you’ll get a posterior distribution of the causal impact.
One idea for measurement in a randomized trial:
In order to apply, you have to list 4 people who would definitely know how awesome you’re being a year from now, and give their contact info. Then, choose 1 of those people 6 months later and 1 person a year later and ask them how awesome the person is being. When you ask, include a “rubric” of various stories of various awesomeness levels, in which the highest levels are not always just $$$ but sometimes are. Ask the people you’re asking to please not contact the person specifically to check awesomeness, because that could introduce bias (“this person is checking, that makes me remember the workshop I did, and feel awesome”).
The 4 people should probably include no couples. Your family, long-term friends...
The one way this breaks down is facebook. I mean, if your interaction with each person is separate, and the workshop makes you seem more awesome to each of 4 people, it is working. But if it just makes you post more upbeat things on Facebook, that might not translate to actual awesomeness. But I think that’s a really minor factor.
Sure, it’s gonna be a noisy and imperfect measurement. You will have to look at standard deviations and calculate power (including burning all 4 contacts for some people to see the within-subject variance). Also, correct for demographic info on contacts, and various other tricks to increase power. But one way or another, you’ll get a posterior distribution of the causal impact.