The fundemental issue of these studies is the reliance on surveys. In most of them, they are the only source of feedback! Why wasn’t there any integration of objective measures like incarceration and hospital visits from the start? This isn’t a minor issue—it’s a flaw in the foundation.
I do not agree that hospital visits and incarceration statistics, (although I’d love to have those numbers), are foundational to measuring the impactfulness of a homeless intervention:
Overwhelming demographic data as well as medical analysis make it evident that living on the streets directly accounts for most, though not all, of the massive mortality rate increase. There is acausal relationship between living on the streets and high death rates, especially in Arizona due to the high summer heat.
I’d be happy to hear why you might disagree, but I believe it’s well-established ^ that having to live on the streets directly results in almost all of the complications, hospital visits, incarceration, and other service costs that are incurred trying to keep homeless people alive. Not the other way around.
Having stable housing is by far the most important objective measure, followed in my opinion by:
Housing Situation (the major determinant of hospitalizations & incarceration)
Employment situation (are they going to fall back on the streets)
Under these, there are several other significant, although not foundational, metrics:
Net Worth (how much of a cash cushion do they keep over time)
Quality of Life
hospitalization & incarceration metrics
Spending Metrics
To be honest, I hadn’t even thought of adding hospitalization and/or incarceration numbers until you brought them up. That said, I will add them to our pool of desired data points.
While I do see your point about surveys maybe not resulting in accurate drug usage numbers, I don’t think they are all that important to the effectiveness of the intervention for the reasons I stated in the last post. They are the only subjective parts of these surveys that might be skewed due to cognitive bias.
The only feedback you are getting is from surveys. They are subjective by definition. Gifts provide immense psychological pressure for reciprocation, especially if it’s presented as “no-strings-attached”. Every metric you cite improvement in is subject to this. This is why I’m so troubled by the lack of objective metrics or any other attempt to mitigate this. The fact that this isn’t addressed is a major red flag. Do you have any statisticians associated with the project to help with study design?
Let me give an analogy. Let’s say you dropped out of college and play video games all day. Your parents call in on the weekends and ask you about your job search and how much video games you’re playing. Do you think you’re more likely to lie or exagerate about your achievements if they’re paying your rent? Even if you knew for sure they weren’t going to pull finiancial support regardless? And what if this continued over the course of months?
I will write up an article some time this week regarding all this.
Our organization is not big enough to hire a statistician, although we will for sure get one when we are able to build a sufficiently large study / program. I’d be happy to refer you to the people that do have a ton of statisticians:
Let’s use a different analogy. Let’s say that you are in exactly the same situation you are in right now, and some random organization decided to start giving you $1,000 checks every month for one year. All they want, is periodic updates on how you’re doing, and they tell you that your answers are anonymized and will not affect the payments. Would you go out of your way to lie to them?
We are not trying to be anyone’s parents, and have no desire for the weird inter-personal shame dynamics that would be going on in your analogy.
The fundemental issue of these studies is the reliance on surveys. In most of them, they are the only source of feedback! Why wasn’t there any integration of objective measures like incarceration and hospital visits from the start? This isn’t a minor issue—it’s a flaw in the foundation.
I do not agree that hospital visits and incarceration statistics, (although I’d love to have those numbers), are foundational to measuring the impactfulness of a homeless intervention:
I’d be happy to hear why you might disagree, but I believe it’s well-established ^ that having to live on the streets directly results in almost all of the complications, hospital visits, incarceration, and other service costs that are incurred trying to keep homeless people alive. Not the other way around.
Having stable housing is by far the most important objective measure, followed in my opinion by:
Housing Situation (the major determinant of hospitalizations & incarceration)
Employment situation (are they going to fall back on the streets)
Under these, there are several other significant, although not foundational, metrics:
Net Worth (how much of a cash cushion do they keep over time)
Quality of Life
hospitalization & incarceration metrics
Spending Metrics
To be honest, I hadn’t even thought of adding hospitalization and/or incarceration numbers until you brought them up. That said, I will add them to our pool of desired data points.
While I do see your point about surveys maybe not resulting in accurate drug usage numbers, I don’t think they are all that important to the effectiveness of the intervention for the reasons I stated in the last post. They are the only subjective parts of these surveys that might be skewed due to cognitive bias.
The only feedback you are getting is from surveys. They are subjective by definition. Gifts provide immense psychological pressure for reciprocation, especially if it’s presented as “no-strings-attached”. Every metric you cite improvement in is subject to this. This is why I’m so troubled by the lack of objective metrics or any other attempt to mitigate this. The fact that this isn’t addressed is a major red flag. Do you have any statisticians associated with the project to help with study design?
Let me give an analogy. Let’s say you dropped out of college and play video games all day. Your parents call in on the weekends and ask you about your job search and how much video games you’re playing. Do you think you’re more likely to lie or exagerate about your achievements if they’re paying your rent? Even if you knew for sure they weren’t going to pull finiancial support regardless? And what if this continued over the course of months?
I will write up an article some time this week regarding all this.
Our organization is not big enough to hire a statistician, although we will for sure get one when we are able to build a sufficiently large study / program. I’d be happy to refer you to the people that do have a ton of statisticians:
https://www.givedirectly.org/research-at-give-directly/
https://basicincome.stanford.edu/research/ubi-visualization/
https://www.penncgir.org/research
Let’s use a different analogy. Let’s say that you are in exactly the same situation you are in right now, and some random organization decided to start giving you $1,000 checks every month for one year. All they want, is periodic updates on how you’re doing, and they tell you that your answers are anonymized and will not affect the payments. Would you go out of your way to lie to them?
We are not trying to be anyone’s parents, and have no desire for the weird inter-personal shame dynamics that would be going on in your analogy.