I have serious concerns about the study designs. Firstly, everything is based on surveys. Saying to someone (implicitly) “I am giving you free money to do less drugs. Now, how much less drugs are you doing?” lacks rigor in a fundamental sense. There needs to be objective follow-up metrics with things like emergency room visits and incarceration rates.
“Nonetheless, spending on drugs & alcohol went down by 39%, and the pilot was a massive success.”
According to surveys given to the participants. Even if you tell them that the payments aren’t contingent on positive results, they don’t necessarily believe you. And even if they do, they’ll still feel obliged to give you the results you’re after. This is a commonly known effect in psychology and sociology, but it was never addressed.
Secondly, there are no comparisons with other programs, like directly providing housing. What exactly is the relative benefit of direct cash transfers over conventional welfare programs like food stamps? Less overhead? It seems marginal at best.
Thirdly, most of your links are to your own foundation, and you’re explicitly here to ask for money. Everything is optimized to make your hypothesis look good without a counterargument. You understand how this looks.
First, thank you for your rigor in analyzing the homelessness part of the post! I most certainly agree with you that cash transfers—explicitly relating to homeless individuals—need more studies and more rigorous RTCs from independent sources.
According to surveys given to the participants. Even if you tell them that the payments aren’t contingent on positive results, they don’t necessarily believe you. And even if they do, they’ll still feel obliged to give you the results you’re after. This is a commonly known effect in psychology and sociology, but it was never addressed.
Thanks for pointing this out, I completely agree. I’ll add this to the document as an important disclaimer.
That said, I also want to add a modifier talking about how the general public is also full of drug users (have you had beer/wine/cannabis in the last year?) and we don’t judge them at all. There’s a big difference between drug use and drug addiction, and there’s a very strong argument that we probably shouldn’t care about drug use metrics at all. It’s incarceration rates & hospital visits that we need more data on. It’s also kinda impossible to accurately determine who does how many drugs without extremely invasive & expensive oversight.
Secondly, there are no comparisons with other programs, like directly providing housing. What exactly is the relative benefit of direct cash transfers over conventional welfare programs like food stamps? Less overhead? It seems marginal at best.
I did try to compare with housing first, especially about the budgets spent per ‘unit’. I think that producing academic studies on these topics is essential over the next few years, I want to keep our work as falsifiable as possible.
Comparisons differ from intervention to intervention, but it comes back to the snake charts that I made right near the top.
It’s not just overhead. Food stamps are pretty similar to guaranteed income because neither requires infrastructure or supply chains (like food banks do), but you still have to account for the loss caused by.
Beneficiary Paperwork: Paperwork is usually an undignified hassle within conventional welfare programs. Wasting beneficiaries’ time is costly, and hiring a ton of bureaucrats to scrutinize people is also costly.
Inefficient Value transfer: Beneficiaries do not get exactly what they need most, so the counterfactual impact would (almost) always be larger if they had cash. I used the example below, which would probably waste ~20% of the value vs the counterfactual.
“Fundamentally, poor people are way better than nonprofit ‘experts’ at knowing what they need and getting it. For example, if someone needs a car loan and the aid program provides food stamps, they have to launder their food stamps to get the cash for their car loan. Guaranteed income empowers poor people with liquid aid.”
Thirdly, most of your links are to your own foundation, and you’re explicitly here to ask for money. Everything is optimized to make your hypothesis look good without a counterargument.
So I primarily linked to my website’s research page because it has all of the links to all of the studies on it. It has a database (not curated by us) of 320+ studies, so I see your point but I’m far more interested in people accessing the research put together in one spot. For homelessness, I linked to the website because you can’t click on the links in the table picture. You can click on the links, and see the source data on the website, and no one else has a list like that. I’m explicitly asking for attention -
And as for your last sentence, yes I’m looking for 3 things.
Someone such as yourself to come up with a fundamental problem with any part of our plan (not just homelessness, as that’s just a tiny part with greater risk but also reward), and save me a bunch of wasted time doing something that won’t lead anywhere.
You’ve hit the nail on the head, after looking at every single piece of research into “What happens when you provide guaranteed income to homeless individuals,” I can’t find any strong counter-arguments, other than there haven’t been enough studies done yet. And considering the unparalleled research into cash transfers on other populations (global poverty, impoverished people in cities, even formerly incarcerated individuals) I’m really not optimizing the links. I’d be happy to see if you can find pretty much any strong evidence that guaranteed income isn’t simply effective.
I did state explicitly that I used a drastic underestimate for all of the impact estimates because of the lack of homelessness RTCs. Luckily there are several on the way, I’m especially looking forward to the one in Denver.
In every experiment to date, guaranteed income has helped at least 66% of participants regain economic stability. We are using an unreasonably low estimate of 50% for two reasons.
There have only been three experiments completed as of November 2022; it’s not the biggest data set.
Because of the logarithmic rate of the success cost curve, we could have miscalculated the optimal duration and/or the monthly amount needed. If we set the bar too high for ourselves and miss slightly, our pilot could look like a failure even if it’s extremely impactful. If only a third or quarter of participants regained housing stability, our pilot would still be an order of magnitude more cost-effective compared to other homelessness interventions
The fundemental issue of these studies is the reliance on surveys. In most of them, they are the only source of feedback! Why wasn’t there any integration of objective measures like incarceration and hospital visits from the start? This isn’t a minor issue—it’s a flaw in the foundation.
I do not agree that hospital visits and incarceration statistics, (although I’d love to have those numbers), are foundational to measuring the impactfulness of a homeless intervention:
Overwhelming demographic data as well as medical analysis make it evident that living on the streets directly accounts for most, though not all, of the massive mortality rate increase. There is acausal relationship between living on the streets and high death rates, especially in Arizona due to the high summer heat.
I’d be happy to hear why you might disagree, but I believe it’s well-established ^ that having to live on the streets directly results in almost all of the complications, hospital visits, incarceration, and other service costs that are incurred trying to keep homeless people alive. Not the other way around.
Having stable housing is by far the most important objective measure, followed in my opinion by:
Housing Situation (the major determinant of hospitalizations & incarceration)
Employment situation (are they going to fall back on the streets)
Under these, there are several other significant, although not foundational, metrics:
Net Worth (how much of a cash cushion do they keep over time)
Quality of Life
hospitalization & incarceration metrics
Spending Metrics
To be honest, I hadn’t even thought of adding hospitalization and/or incarceration numbers until you brought them up. That said, I will add them to our pool of desired data points.
While I do see your point about surveys maybe not resulting in accurate drug usage numbers, I don’t think they are all that important to the effectiveness of the intervention for the reasons I stated in the last post. They are the only subjective parts of these surveys that might be skewed due to cognitive bias.
The only feedback you are getting is from surveys. They are subjective by definition. Gifts provide immense psychological pressure for reciprocation, especially if it’s presented as “no-strings-attached”. Every metric you cite improvement in is subject to this. This is why I’m so troubled by the lack of objective metrics or any other attempt to mitigate this. The fact that this isn’t addressed is a major red flag. Do you have any statisticians associated with the project to help with study design?
Let me give an analogy. Let’s say you dropped out of college and play video games all day. Your parents call in on the weekends and ask you about your job search and how much video games you’re playing. Do you think you’re more likely to lie or exagerate about your achievements if they’re paying your rent? Even if you knew for sure they weren’t going to pull finiancial support regardless? And what if this continued over the course of months?
I will write up an article some time this week regarding all this.
Our organization is not big enough to hire a statistician, although we will for sure get one when we are able to build a sufficiently large study / program. I’d be happy to refer you to the people that do have a ton of statisticians:
Let’s use a different analogy. Let’s say that you are in exactly the same situation you are in right now, and some random organization decided to start giving you $1,000 checks every month for one year. All they want, is periodic updates on how you’re doing, and they tell you that your answers are anonymized and will not affect the payments. Would you go out of your way to lie to them?
We are not trying to be anyone’s parents, and have no desire for the weird inter-personal shame dynamics that would be going on in your analogy.
I have serious concerns about the study designs. Firstly, everything is based on surveys. Saying to someone (implicitly) “I am giving you free money to do less drugs. Now, how much less drugs are you doing?” lacks rigor in a fundamental sense. There needs to be objective follow-up metrics with things like emergency room visits and incarceration rates.
“Nonetheless, spending on drugs & alcohol went down by 39%, and the pilot was a massive success.”
According to surveys given to the participants. Even if you tell them that the payments aren’t contingent on positive results, they don’t necessarily believe you. And even if they do, they’ll still feel obliged to give you the results you’re after. This is a commonly known effect in psychology and sociology, but it was never addressed.
Secondly, there are no comparisons with other programs, like directly providing housing. What exactly is the relative benefit of direct cash transfers over conventional welfare programs like food stamps? Less overhead? It seems marginal at best.
Thirdly, most of your links are to your own foundation, and you’re explicitly here to ask for money. Everything is optimized to make your hypothesis look good without a counterargument. You understand how this looks.
First, thank you for your rigor in analyzing the homelessness part of the post! I most certainly agree with you that cash transfers—explicitly relating to homeless individuals—need more studies and more rigorous RTCs from independent sources.
Thanks for pointing this out, I completely agree. I’ll add this to the document as an important disclaimer.
That said, I also want to add a modifier talking about how the general public is also full of drug users (have you had beer/wine/cannabis in the last year?) and we don’t judge them at all. There’s a big difference between drug use and drug addiction, and there’s a very strong argument that we probably shouldn’t care about drug use metrics at all. It’s incarceration rates & hospital visits that we need more data on. It’s also kinda impossible to accurately determine who does how many drugs without extremely invasive & expensive oversight.
I did try to compare with housing first, especially about the budgets spent per ‘unit’. I think that producing academic studies on these topics is essential over the next few years, I want to keep our work as falsifiable as possible.
Comparisons differ from intervention to intervention, but it comes back to the snake charts that I made right near the top.
It’s not just overhead. Food stamps are pretty similar to guaranteed income because neither requires infrastructure or supply chains (like food banks do), but you still have to account for the loss caused by.
Beneficiary Paperwork: Paperwork is usually an undignified hassle within conventional welfare programs. Wasting beneficiaries’ time is costly, and hiring a ton of bureaucrats to scrutinize people is also costly.
Inefficient Value transfer: Beneficiaries do not get exactly what they need most, so the counterfactual impact would (almost) always be larger if they had cash. I used the example below, which would probably waste ~20% of the value vs the counterfactual.
So I primarily linked to my website’s research page because it has all of the links to all of the studies on it. It has a database (not curated by us) of 320+ studies, so I see your point but I’m far more interested in people accessing the research put together in one spot. For homelessness, I linked to the website because you can’t click on the links in the table picture. You can click on the links, and see the source data on the website, and no one else has a list like that. I’m explicitly asking for attention -
And as for your last sentence, yes I’m looking for 3 things.
Someone such as yourself to come up with a fundamental problem with any part of our plan (not just homelessness, as that’s just a tiny part with greater risk but also reward), and save me a bunch of wasted time doing something that won’t lead anywhere.
You’ve hit the nail on the head, after looking at every single piece of research into “What happens when you provide guaranteed income to homeless individuals,” I can’t find any strong counter-arguments, other than there haven’t been enough studies done yet. And considering the unparalleled research into cash transfers on other populations (global poverty, impoverished people in cities, even formerly incarcerated individuals) I’m really not optimizing the links. I’d be happy to see if you can find pretty much any strong evidence that guaranteed income isn’t simply effective.
I did state explicitly that I used a drastic underestimate for all of the impact estimates because of the lack of homelessness RTCs. Luckily there are several on the way, I’m especially looking forward to the one in Denver.
The fundemental issue of these studies is the reliance on surveys. In most of them, they are the only source of feedback! Why wasn’t there any integration of objective measures like incarceration and hospital visits from the start? This isn’t a minor issue—it’s a flaw in the foundation.
I do not agree that hospital visits and incarceration statistics, (although I’d love to have those numbers), are foundational to measuring the impactfulness of a homeless intervention:
I’d be happy to hear why you might disagree, but I believe it’s well-established ^ that having to live on the streets directly results in almost all of the complications, hospital visits, incarceration, and other service costs that are incurred trying to keep homeless people alive. Not the other way around.
Having stable housing is by far the most important objective measure, followed in my opinion by:
Housing Situation (the major determinant of hospitalizations & incarceration)
Employment situation (are they going to fall back on the streets)
Under these, there are several other significant, although not foundational, metrics:
Net Worth (how much of a cash cushion do they keep over time)
Quality of Life
hospitalization & incarceration metrics
Spending Metrics
To be honest, I hadn’t even thought of adding hospitalization and/or incarceration numbers until you brought them up. That said, I will add them to our pool of desired data points.
While I do see your point about surveys maybe not resulting in accurate drug usage numbers, I don’t think they are all that important to the effectiveness of the intervention for the reasons I stated in the last post. They are the only subjective parts of these surveys that might be skewed due to cognitive bias.
The only feedback you are getting is from surveys. They are subjective by definition. Gifts provide immense psychological pressure for reciprocation, especially if it’s presented as “no-strings-attached”. Every metric you cite improvement in is subject to this. This is why I’m so troubled by the lack of objective metrics or any other attempt to mitigate this. The fact that this isn’t addressed is a major red flag. Do you have any statisticians associated with the project to help with study design?
Let me give an analogy. Let’s say you dropped out of college and play video games all day. Your parents call in on the weekends and ask you about your job search and how much video games you’re playing. Do you think you’re more likely to lie or exagerate about your achievements if they’re paying your rent? Even if you knew for sure they weren’t going to pull finiancial support regardless? And what if this continued over the course of months?
I will write up an article some time this week regarding all this.
Our organization is not big enough to hire a statistician, although we will for sure get one when we are able to build a sufficiently large study / program. I’d be happy to refer you to the people that do have a ton of statisticians:
https://www.givedirectly.org/research-at-give-directly/
https://basicincome.stanford.edu/research/ubi-visualization/
https://www.penncgir.org/research
Let’s use a different analogy. Let’s say that you are in exactly the same situation you are in right now, and some random organization decided to start giving you $1,000 checks every month for one year. All they want, is periodic updates on how you’re doing, and they tell you that your answers are anonymized and will not affect the payments. Would you go out of your way to lie to them?
We are not trying to be anyone’s parents, and have no desire for the weird inter-personal shame dynamics that would be going on in your analogy.