I guess I just don’t have a strong sense of where the practitioners’ numbers are coming from, or why they believe what they believe. Which is fine if you want to bulid a pipeline that turn some intuitions into decisions, but not obviously incredibly useful for the rest of us (beyond just telling us those intuitions).
Finally, it’s worth mentioning that the cost-effectiveness of these programs relative to one another do not rely very heavily on conversions.
The thing you link shows that if you change the conversion ratio of both programs the same amount, the relative cost-effectiveness doesn’t change, which makes sense. But if workshops produced 100x more conversions than socials, or vice versa, presumably this must make a difference. If you say that the treatment effects only differ by a factor of 2, then fair enough, but that’s just not super a priori clear (and the fact that you claim that (a) you can measure the TDC better and (b) the TDC has a different treatment effect makes me skeptical).
(For the record, I couldn’t really make heads or tails of the spreadsheet you linked or what the calculations in the script were supposed to be, but I didn’t try super hard to understand them—perhaps I’d write something different if I really understood them)
I guess I just don’t have a strong sense of where the practitioners’ numbers are coming from, or why they believe what they believe. Which is fine if you want to bulid a pipeline that turn some intuitions into decisions, but not obviously incredibly useful for the rest of us (beyond just telling us those intuitions).
The thing you link shows that if you change the conversion ratio of both programs the same amount, the relative cost-effectiveness doesn’t change, which makes sense. But if workshops produced 100x more conversions than socials, or vice versa, presumably this must make a difference. If you say that the treatment effects only differ by a factor of 2, then fair enough, but that’s just not super a priori clear (and the fact that you claim that (a) you can measure the TDC better and (b) the TDC has a different treatment effect makes me skeptical).
(For the record, I couldn’t really make heads or tails of the spreadsheet you linked or what the calculations in the script were supposed to be, but I didn’t try super hard to understand them—perhaps I’d write something different if I really understood them)