And the paper you linked showed that it wasn’t being done for most of Google’s history.
This is a nitpick, but 2000-2007 (the period between when AdWords launched and when the paper says they started quantitative ad blindness research) is 1⁄3 of Google’s history, not “most”.
I’m also not sure if the experiments could have been run much earlier, because I’m not sure identity was stable enough before users were signing into search pages.
Also, this sort of optimization isn’t that valuable compared to much bigger opportunities for growth they had in the early 2000s.
If Google doesn’t do it, I would be doubtful if anyone, even a peer like Amazon, does.
Why are you saying Google doesn’t do it? I understand arguing about whether Google was doing it at various times, whether they should have prioritized it more highly, etc, but it’s clearly used and I’ve talked to people who work on it.
Would you be interested in betting on whether Amazon has quantified the effects of ad blindness? I think we could probably find an Amazon employee to verify.
Which is just another way of saying that before then they hadn’t used their long-term value measurements to figure out what threshold of ads to run before. Whether 2015 or 2013, this is damning.
It’s specifically about mobile, which in 2013 was only about 10% of traffic and much less by monetization. Similar desktop experiments had been run earlier.
But I also think you’re misinterpreting the paper to be about “how many ads should we run” and that those launches simply reduced the number of ads they were running. I’m claiming that the tuning of how many ads to run to maximize long-term value was already pretty good by 2013, but having a better experimental framework allowed them to increase long-term value by figuring out which specific kinds of ads to run or not run. As a rough example (from my head, I haven’t looked at these launches) imagine an advertiser is willing to pay you a lot to run a bad ad that makes people pay less attention to your ads overall. If you turn down your threshold for how many ads to show, this bad ad will still get through. Measuring this kind of negative externality that varies on a per-ad basis is really hard, and it’s especially hard if you have to run very long experiments to quantify the effect. One of the powerful tools in the paper is estimating long-term impacts from short term metrics so you can iterate faster, which makes it easier to evaluate many things including these kind of externalities.
(As before, speaking only for myself and not for Google)
This is a nitpick, but 2000-2007 (the period between when AdWords launched and when the paper says they started quantitative ad blindness research) is 1⁄3 of Google’s history, not “most”.
I’m also not sure if the experiments could have been run much earlier, because I’m not sure identity was stable enough before users were signing into search pages.
Also, this sort of optimization isn’t that valuable compared to much bigger opportunities for growth they had in the early 2000s.
Why are you saying Google doesn’t do it? I understand arguing about whether Google was doing it at various times, whether they should have prioritized it more highly, etc, but it’s clearly used and I’ve talked to people who work on it.
Would you be interested in betting on whether Amazon has quantified the effects of ad blindness? I think we could probably find an Amazon employee to verify.
It’s specifically about mobile, which in 2013 was only about 10% of traffic and much less by monetization. Similar desktop experiments had been run earlier.
But I also think you’re misinterpreting the paper to be about “how many ads should we run” and that those launches simply reduced the number of ads they were running. I’m claiming that the tuning of how many ads to run to maximize long-term value was already pretty good by 2013, but having a better experimental framework allowed them to increase long-term value by figuring out which specific kinds of ads to run or not run. As a rough example (from my head, I haven’t looked at these launches) imagine an advertiser is willing to pay you a lot to run a bad ad that makes people pay less attention to your ads overall. If you turn down your threshold for how many ads to show, this bad ad will still get through. Measuring this kind of negative externality that varies on a per-ad basis is really hard, and it’s especially hard if you have to run very long experiments to quantify the effect. One of the powerful tools in the paper is estimating long-term impacts from short term metrics so you can iterate faster, which makes it easier to evaluate many things including these kind of externalities.
(As before, speaking only for myself and not for Google)