I’m sure you took into account things like house price appreciation. But what you said about investing the difference between what renters pay and what owners pay was misleading and wrong. You can do as careful and accurate and insightful a simulation as you please, but if what you say is “people pay more in mortgage + fees + taxes than in rent, which is bad because they could have invested the difference in the market and then they’d have got some actual returns” or “rent is bad because it’s just throwing money away” then you are making a broken argument and I think you shouldn’t do that. Not even if the conclusion of the broken argument happens to be the same as the conclusion of the careful accurate insightful simulation.
The question isn’t whether back-testing is hard, it’s how well you did it and whether whatever assumptions you made seem reasonable to any given reader. Again, my complaint isn’t that your final results are bad, it’s that we have no way of telling whether your final results are good or bad because you didn’t show us any of the information we’d need to decide.
[EDITED to add:] This is all coming out more aggressive-sounding than I would like, and I hope I’m not giving the impression that I think the OP is terrible or that you’re a bad person or any such thing. It just seems as if your responses to my comments aren’t engaging with the actual points those comments are trying to make. That may be wholly or partly because of lack of clarity in what I’ve written, in which case please accept my apologies.
I get it, I get this shape of feedback on LW and expect it. I think it’s a product of differing expectations for what sort of posts belong on LW. This post isn’t particularly concerned with the full suite of tools you’d need to deploy to rigorously show that my argument is correct. And that isn’t its intention. I wrote this because people asked me to surface all the relevant considerations I was aware of for home purchase. In my model, becoming aware of the relevant considerations is often much more important than tuning the confidence bounds of the considerations one is already aware of. The frame of the post assumes the sign flip that I think those considerations warrant, which makes it not come off as a collaborative truth seeking effort either. And I am open to having missed important things. But I’m not going to backfill all the epistemic rigor. I don’t know of a good write up that argues the opposite side of this that actually takes into account the seemingly important things.
This post isn’t particularly concerned with the full suite of tools you’d need to deploy to rigorously show that my argument is correct. And that isn’t its intention. I wrote this because people asked me to surface all the relevant considerations I was aware of for home purchase.
I haven’t been following the thread here, but from a quick glance, seems like that would have been good to include an epistemic status or other preamble about your intention with the post. From my quick read, I had the impression that it was very carefully considered/rigorous.
Added some info at the top. I think some people want a degree of quantitative modeling that I’m not going to pour 10+ hours into polishing into usable form. This should ultimately really be a guesstimate model and I’d encourage someone with the relevant chops to apply to the EA funds to get a grant to create a much better rent-to-buy calculator that the entire community can then use to get better confidence bounds on individual decisions.
My own take is “it’s quite important for people to throw up ideas on LessWrong without being forced to delve into hours of rigor, but for that to really work as a norm I think it’s fairly important to have epistemic statuses spelling that out, so people know what to expect.”
(I also have not read this post or discussion, don’t have strong opinions about this particular example).
I’m sure you took into account things like house price appreciation. But what you said about investing the difference between what renters pay and what owners pay was misleading and wrong. You can do as careful and accurate and insightful a simulation as you please, but if what you say is “people pay more in mortgage + fees + taxes than in rent, which is bad because they could have invested the difference in the market and then they’d have got some actual returns” or “rent is bad because it’s just throwing money away” then you are making a broken argument and I think you shouldn’t do that. Not even if the conclusion of the broken argument happens to be the same as the conclusion of the careful accurate insightful simulation.
The question isn’t whether back-testing is hard, it’s how well you did it and whether whatever assumptions you made seem reasonable to any given reader. Again, my complaint isn’t that your final results are bad, it’s that we have no way of telling whether your final results are good or bad because you didn’t show us any of the information we’d need to decide.
[EDITED to add:] This is all coming out more aggressive-sounding than I would like, and I hope I’m not giving the impression that I think the OP is terrible or that you’re a bad person or any such thing. It just seems as if your responses to my comments aren’t engaging with the actual points those comments are trying to make. That may be wholly or partly because of lack of clarity in what I’ve written, in which case please accept my apologies.
I get it, I get this shape of feedback on LW and expect it. I think it’s a product of differing expectations for what sort of posts belong on LW. This post isn’t particularly concerned with the full suite of tools you’d need to deploy to rigorously show that my argument is correct. And that isn’t its intention. I wrote this because people asked me to surface all the relevant considerations I was aware of for home purchase. In my model, becoming aware of the relevant considerations is often much more important than tuning the confidence bounds of the considerations one is already aware of. The frame of the post assumes the sign flip that I think those considerations warrant, which makes it not come off as a collaborative truth seeking effort either. And I am open to having missed important things. But I’m not going to backfill all the epistemic rigor. I don’t know of a good write up that argues the opposite side of this that actually takes into account the seemingly important things.
I haven’t been following the thread here, but from a quick glance, seems like that would have been good to include an epistemic status or other preamble about your intention with the post. From my quick read, I had the impression that it was very carefully considered/rigorous.
Added some info at the top. I think some people want a degree of quantitative modeling that I’m not going to pour 10+ hours into polishing into usable form. This should ultimately really be a guesstimate model and I’d encourage someone with the relevant chops to apply to the EA funds to get a grant to create a much better rent-to-buy calculator that the entire community can then use to get better confidence bounds on individual decisions.
My own take is “it’s quite important for people to throw up ideas on LessWrong without being forced to delve into hours of rigor, but for that to really work as a norm I think it’s fairly important to have epistemic statuses spelling that out, so people know what to expect.”
(I also have not read this post or discussion, don’t have strong opinions about this particular example).
Yes, with the edit “Quite important” → “necessary”