The “don’t have to be personalized” part of your argument rests on the “4% less revenue” statistic from Marotta et al. (2019), but that seems to be a very bad source. Here’s Garrett Johnson’s literature review slide: https://twitter.com/garjoh_canuck/status/1318989360407236609.
Of the six studies, five (from academia, industry, and government sources) come to the conclusion that removing personalization would cost websites somewhere between 50% and 70% of their revenue, and one concludes 4%. Moreover, the Marotta paper rests on subtle statistical methods aimed at the problem “We have a lot of observational data but we can’t run an A/B experiment, so let’s (use augmented inverse probability weighting to) figure out what would happen if running the direct experiment were possible.” That’s a hard but admirable goal. But in the Google paper they actually did run exactly that A/B experiment, the very same one that Marotta et al were trying to predict, and they got 52% rev loss instead.
There are good reasons to be skeptical of the 50%-70% range — in particular those measurements are all happening in an environment where personalized advertising channels still exist, so they don’t get at global equilibrium after the forced behavior changes that would come from personalization going away across the board. From observing what happened to ads prices on iOS in recent years, I’d say it seems very plausible that the stable outcome would be more like 40% instead of 50%-70%.
But the 4% number is quite unsupportable given the data at hand. And telling most websites in the EU that they will lose ~half of their revenue is not an appealing prospect.
Good points, I’ll look into the other studies at another time. I remember a german newspaper actually switching completely to non-targeted ads after their own experiment, but can’t find the source anymore. I’ll comment it here, if I find it again.
Thanks especially for your transparency on your Motivation and Disclaimer.
The “don’t have to be personalized” part of your argument rests on the “4% less revenue” statistic from Marotta et al. (2019), but that seems to be a very bad source. Here’s Garrett Johnson’s literature review slide: https://twitter.com/garjoh_canuck/status/1318989360407236609.
Of the six studies, five (from academia, industry, and government sources) come to the conclusion that removing personalization would cost websites somewhere between 50% and 70% of their revenue, and one concludes 4%. Moreover, the Marotta paper rests on subtle statistical methods aimed at the problem “We have a lot of observational data but we can’t run an A/B experiment, so let’s (use augmented inverse probability weighting to) figure out what would happen if running the direct experiment were possible.” That’s a hard but admirable goal. But in the Google paper they actually did run exactly that A/B experiment, the very same one that Marotta et al were trying to predict, and they got 52% rev loss instead.
There are good reasons to be skeptical of the 50%-70% range — in particular those measurements are all happening in an environment where personalized advertising channels still exist, so they don’t get at global equilibrium after the forced behavior changes that would come from personalization going away across the board. From observing what happened to ads prices on iOS in recent years, I’d say it seems very plausible that the stable outcome would be more like 40% instead of 50%-70%.
Motivation&Disclaimer: I’m https://mathstodon.xyz/@Log3overLog2, and I work at Google on the Chrome & Android effort to move to a much more private way to do ads personalization. So I’m deeply vested in this question.
But the 4% number is quite unsupportable given the data at hand. And telling most websites in the EU that they will lose ~half of their revenue is not an appealing prospect.
Good points, I’ll look into the other studies at another time. I remember a german newspaper actually switching completely to non-targeted ads after their own experiment, but can’t find the source anymore. I’ll comment it here, if I find it again.
Thanks especially for your transparency on your Motivation and Disclaimer.