Funny enough, I’ve recently been contemplating asking a question here on LW like “what actually-quite-good youtube channels are out there?” precisely because I suspected there were a lot of hidden gems like you mention! I didn’t get around to it, though, perhaps because watching youtube feels low status so I felt like it would be embarassing to visibly put effort into optimizing my youtube-watching.
Anyway, there are a few reasons why I thought this was a priori plausible:
First and foremost, I see this as an example of the tails come apart (ie Goodhart). Even if quality and reception are very highly correlated, you should expect the top of the two to be different, based on noise alone. But maybe that’s not satisfying: what’s driving the noise? Why isn’t reception exactly proportional to quality?
A big factor here, for a lot of cases, is going to be network effects. Small fluctuations in popularity don’t get corrected, they get exacerbated. Two people can release similarly great videos, and the fact that one gets its first views from the right people (who show all their friends) can snowball into a big lead. And this can snowball into making one channel a huge success while the other practically-equivalent channel is lost in obscurity.
But there are also going to be a lot of “noise” factors that aren’t random at all. EG, how clickbaity a video is. Out of two similar videos, if one is a little more informative but the other is a little more clickbaity, I expect the clickbaity one to do better.
The third point means we could see the tails come apart to a very high degree, even if there’s no noise in the system at all. To possibilities like #3, you respond:
I want to focus on my core point here. DH6, not DH5. You might be able to make points about how being brilliant doesn’t necessarily make you good on camera, or about how marketing and promotion matters a lot, or about releasing videos on a consistent schedule, or about how maybe they’re not producing the types of content YouTube viewers want to watch. Some of those might be fair points, but do you think that they refute my core point? My core point is that there are people like DHH, like Massimo, who should be getting way more views on their videos, given the quality of the content they’re producing and what they’ve done to promote that content. Do you disagree with that?
(I don’t disagree at all, but) I think you’re implicitly assuming something like: even if DHH is a little less clickbaity than competitors, this should only translate to a somewhat cooler reception. This is reinforced by your expectation that:
If my quality is, say, a 7⁄10, my level of success should be somewhere in that ballpark. Maybe the market would be inefficient and I’d only reach a 5⁄10 or a 4⁄10. Or maybe I’d be lucky and reach an 8⁄10 or 9⁄10.
But reception can be highly nonlinear, perhaps closer to exponential. So, at the upper ends of quality, small differences in quality lead to very large differences in reception.
We could see something like:
Quality Reception
1⁄100⁄10
2⁄100⁄10
3⁄100⁄10
4⁄100⁄10
5⁄100⁄10
6⁄100⁄10
7⁄101⁄10
8⁄102⁄10
9⁄105⁄10
10⁄1010⁄10
Again, even without randomness, in a system like this you could see the effects you’re describing.
Funny enough, I’ve recently been contemplating asking a question here on LW like “what actually-quite-good youtube channels are out there?” precisely because I suspected there were a lot of hidden gems like you mention! I didn’t get around to it, though, perhaps because watching youtube feels low status so I felt like it would be embarassing to visibly put effort into optimizing my youtube-watching.
Also I think you might really like Eric Normand. In particular the videos/podcasts where he spends 60-120 minutes exploring, distilling and discussing famous papers in computer science. It makes me sad that these videos only get a couple hundred views.
And if you happen to be into basketball/are curious about what the smart basketball people sound like, I have a special place in my heart for Thinking Basketball. If you end up watching any videos, I’d be interested to hear what you think of them! Especially if you aren’t a sports person, and especially if you didn’t like them :)
(I don’t disagree at all, but)
Overall, I get the impression that we don’t disagree about anything, and that I was a little misleading in my OP.
I think you’re implicitly assuming something like: even if DHH is a little less clickbaity than competitors, this should only translate to a somewhat cooler reception.
I don’t think that’s what it is. Let me try to clarify. I agree that the “noise” factors you mention all matter in addition to “raw quality”. Things like click-baitiness, production quality, personality, etc. Let’s operationally define “quality” to be some metric that encapsulates “noise factors” + “raw quality”. My judgement is that… let’s say DHH’s videos are a 7⁄10 in this metric. He’s getting 30k views when others who I judge to be significantly worse in this “quality” metric, maybe a 3⁄10, that encapsulates a variety of factors, are getting 300k views or even 3M views. 7⁄10 quality → 3⁄10 reception, and 3⁄10 quality → 7⁄10 reception.
I think that what you propose here is a very plausible explanation for this:
A big factor here, for a lot of cases, is going to be network effects. Small fluctuations in popularity don’t get corrected, they get exacerbated. Two people can release similarly great videos, and the fact that one gets its first views from the right people (who show all their friends) can snowball into a big lead. And this can snowball into making one channel a huge success while the other practically-equivalent channel is lost in obscurity.
I regret using the YouTube examples. I think a much better example is what I’ve seen watching Chefs Table where one lucky break such as being ranked in a magazine produces a snowball effect that takes someone like Massimo from obscurity to international acclaim shockingly quickly. I suspect that similar things happen all the time in fields like music, writing, arts, and business. Back to Massimo’s restaurant: things like location, service, lighting, and ambiance (“noise factors”) matter in addition to food quality (“raw quality”). But whether or not you rise to the top seems to often depend on that lucky break + snowball effect.
My point isn’t about noise factors vs raw quality, or about linear vs exponential relationships between quality and reception. It’s about significant inefficiencies and lucky breaks.
Well actually, I think I’m having trouble articulating what my point is. So think of the above as a WIP.
Funny enough, I’ve recently been contemplating asking a question here on LW like “what actually-quite-good youtube channels are out there?” precisely because I suspected there were a lot of hidden gems like you mention! I didn’t get around to it, though, perhaps because watching youtube feels low status so I felt like it would be embarassing to visibly put effort into optimizing my youtube-watching.
Anyway, there are a few reasons why I thought this was a priori plausible:
First and foremost, I see this as an example of the tails come apart (ie Goodhart). Even if quality and reception are very highly correlated, you should expect the top of the two to be different, based on noise alone. But maybe that’s not satisfying: what’s driving the noise? Why isn’t reception exactly proportional to quality?
A big factor here, for a lot of cases, is going to be network effects. Small fluctuations in popularity don’t get corrected, they get exacerbated. Two people can release similarly great videos, and the fact that one gets its first views from the right people (who show all their friends) can snowball into a big lead. And this can snowball into making one channel a huge success while the other practically-equivalent channel is lost in obscurity.
But there are also going to be a lot of “noise” factors that aren’t random at all. EG, how clickbaity a video is. Out of two similar videos, if one is a little more informative but the other is a little more clickbaity, I expect the clickbaity one to do better.
The third point means we could see the tails come apart to a very high degree, even if there’s no noise in the system at all. To possibilities like #3, you respond:
(I don’t disagree at all, but) I think you’re implicitly assuming something like: even if DHH is a little less clickbaity than competitors, this should only translate to a somewhat cooler reception. This is reinforced by your expectation that:
But reception can be highly nonlinear, perhaps closer to exponential. So, at the upper ends of quality, small differences in quality lead to very large differences in reception.
We could see something like:
Quality Reception
1⁄10 0⁄10
2⁄10 0⁄10
3⁄10 0⁄10
4⁄10 0⁄10
5⁄10 0⁄10
6⁄10 0⁄10
7⁄10 1⁄10
8⁄10 2⁄10
9⁄10 5⁄10
10⁄10 10⁄10
Again, even without randomness, in a system like this you could see the effects you’re describing.
Kenji!
Also I think you might really like Eric Normand. In particular the videos/podcasts where he spends 60-120 minutes exploring, distilling and discussing famous papers in computer science. It makes me sad that these videos only get a couple hundred views.
And if you happen to be into basketball/are curious about what the smart basketball people sound like, I have a special place in my heart for Thinking Basketball. If you end up watching any videos, I’d be interested to hear what you think of them! Especially if you aren’t a sports person, and especially if you didn’t like them :)
Overall, I get the impression that we don’t disagree about anything, and that I was a little misleading in my OP.
I don’t think that’s what it is. Let me try to clarify. I agree that the “noise” factors you mention all matter in addition to “raw quality”. Things like click-baitiness, production quality, personality, etc. Let’s operationally define “quality” to be some metric that encapsulates “noise factors” + “raw quality”. My judgement is that… let’s say DHH’s videos are a 7⁄10 in this metric. He’s getting 30k views when others who I judge to be significantly worse in this “quality” metric, maybe a 3⁄10, that encapsulates a variety of factors, are getting 300k views or even 3M views. 7⁄10 quality → 3⁄10 reception, and 3⁄10 quality → 7⁄10 reception.
I think that what you propose here is a very plausible explanation for this:
I regret using the YouTube examples. I think a much better example is what I’ve seen watching Chefs Table where one lucky break such as being ranked in a magazine produces a snowball effect that takes someone like Massimo from obscurity to international acclaim shockingly quickly. I suspect that similar things happen all the time in fields like music, writing, arts, and business. Back to Massimo’s restaurant: things like location, service, lighting, and ambiance (“noise factors”) matter in addition to food quality (“raw quality”). But whether or not you rise to the top seems to often depend on that lucky break + snowball effect.
My point isn’t about noise factors vs raw quality, or about linear vs exponential relationships between quality and reception. It’s about significant inefficiencies and lucky breaks.
Well actually, I think I’m having trouble articulating what my point is. So think of the above as a WIP.