The only feedback you are getting is from surveys. They are subjective by definition. Gifts provide immense psychological pressure for reciprocation, especially if it’s presented as “no-strings-attached”. Every metric you cite improvement in is subject to this. This is why I’m so troubled by the lack of objective metrics or any other attempt to mitigate this. The fact that this isn’t addressed is a major red flag. Do you have any statisticians associated with the project to help with study design?
Let me give an analogy. Let’s say you dropped out of college and play video games all day. Your parents call in on the weekends and ask you about your job search and how much video games you’re playing. Do you think you’re more likely to lie or exagerate about your achievements if they’re paying your rent? Even if you knew for sure they weren’t going to pull finiancial support regardless? And what if this continued over the course of months?
I will write up an article some time this week regarding all this.
Our organization is not big enough to hire a statistician, although we will for sure get one when we are able to build a sufficiently large study / program. I’d be happy to refer you to the people that do have a ton of statisticians:
Let’s use a different analogy. Let’s say that you are in exactly the same situation you are in right now, and some random organization decided to start giving you $1,000 checks every month for one year. All they want, is periodic updates on how you’re doing, and they tell you that your answers are anonymized and will not affect the payments. Would you go out of your way to lie to them?
We are not trying to be anyone’s parents, and have no desire for the weird inter-personal shame dynamics that would be going on in your analogy.
The only feedback you are getting is from surveys. They are subjective by definition. Gifts provide immense psychological pressure for reciprocation, especially if it’s presented as “no-strings-attached”. Every metric you cite improvement in is subject to this. This is why I’m so troubled by the lack of objective metrics or any other attempt to mitigate this. The fact that this isn’t addressed is a major red flag. Do you have any statisticians associated with the project to help with study design?
Let me give an analogy. Let’s say you dropped out of college and play video games all day. Your parents call in on the weekends and ask you about your job search and how much video games you’re playing. Do you think you’re more likely to lie or exagerate about your achievements if they’re paying your rent? Even if you knew for sure they weren’t going to pull finiancial support regardless? And what if this continued over the course of months?
I will write up an article some time this week regarding all this.
Our organization is not big enough to hire a statistician, although we will for sure get one when we are able to build a sufficiently large study / program. I’d be happy to refer you to the people that do have a ton of statisticians:
https://www.givedirectly.org/research-at-give-directly/
https://basicincome.stanford.edu/research/ubi-visualization/
https://www.penncgir.org/research
Let’s use a different analogy. Let’s say that you are in exactly the same situation you are in right now, and some random organization decided to start giving you $1,000 checks every month for one year. All they want, is periodic updates on how you’re doing, and they tell you that your answers are anonymized and will not affect the payments. Would you go out of your way to lie to them?
We are not trying to be anyone’s parents, and have no desire for the weird inter-personal shame dynamics that would be going on in your analogy.