I wrote up a longer, conceptual review. But I also did a brief data collection, which I’ll post here as others might like to build on or go through a similar exercise.
In 2019 YC released a list of their top 100 portfolio companies ranked by valuation and exit size, where applicable.
So I went through the top 50 companies on this list, and gave each company a ranking ranging from −2 for “Very approval-extracting” to 2 for “Very production-oriented”.
To decide on that number, I asked myself questions like “Would growth of this company seem cancerous?” and “Would I reflectively endorse using this product?”
Companies that scored highly include Doordash, Dropbox and Gusto (all 2′s), and companies that score low include Scale.com (which builds tooling to speed up AI research) and Twitch (-2 and −1).
For comparison, I also did the same exercise with the top 50 S&P500 companies by market cap, with high-scoring ones including Microsoft and Visa, and low-scoring ones including Coca Cola and Salesforce.
This scale is Very Made-up and Maybe Useless. But, if nothing else, it seemed like a useful way to get grounded in some data before thinking further about the post.
Overall, the distributions ended up very similar, though YC did come out with a higher mean, mostly driven by fewer negative tail companies.
I did the first 20 from each column of your spreadsheet, and got a different result. I hid your answers before writing mine. My rubric was different; instead of focusing on social value, I focused on what type of business relations a company has. You can see my answers here. These are all very noisy, and I’m not entirely confident I didn’t have rating-drift between when I did the YC ones and when I did the S&P ones, but I got a slightly higher score for S&P companies.
In my rubric, things that mean low scores:
You have a Compliance department
A significant portion of your business is oriented towards appeasing gatekeepers (as opposed to there being no relevant gatekeepers, or fighting them not on their normal terms)
The price is not disclosed until you talk to a salesperson
You need your business partners but they don’t need you
Things that mean high scores:
You are creating a new market
Dealing with gatekeepers is not a major concern
Your customers take your price or leave it
You do not operate a call center
Your business partners need you but you don’t need them
We had maximally-different scores (2 vs −2) for Gusto, Microsoft, Facebook, Visa, Mastercard and PayPal. The correlation between our scores was 0.6 for the YC companies, −0.16 for the S&P 500 companies.
This was a great idea, but I think the spreadsheet fails on two fronts—first, it’s measuring the end product rather than the founders and how they operate and attempt to scale, which is the primary thing Benquo is talking about here I believe, and two is that if I ranked these companies I don’t think there would be that much correlation with these rankings.
In the examples from the comment, and judging purely on nature of product since I don’t know the founders or early histories much, I’d have had Twitch as positive while I had Doordash as negative, I’d agree with Dropbox and Gusto, and Scale is a weird case where we think the product is bad if it is real but that’s orthogonal to the main point here.
Looking at the S&P 500 I see the same thing. Amazon at 0 seems insane to me (I’d be +lots) and McDonalds at −2 even more so especially in its early days (The Founder is a very good movie about its origins).
When I read this essay in 2019, I remember getting the impression that approval-extracting vs production-oriented was supposed to be about the behavior of the founders, not the industry the company competes in.
Companies that scored highly include Doordash, Dropbox and Gusto (all 2′s), and companies that score low include Scale.com and Twitch (-2 and −1).
I can’t quite tell why you think Twitch is bad. It is subject to network effects, kind of a social media company, is that why? And I don’t know what Scale.com is other than some AI company.
For many of these companies I feel like my opinion changes as they become monopolies. For example, we use Gusto at LW, it’s great. That said, if it became the primary company people used in a country to interact with a part of government, then I could imagine Gusto working with that government to extract money from people in some way. So I like it to a point, then suddenly I might really not like it.
Overall, the distributions ended up very similar, though YC did come out with a higher mean, mostly driven by fewer negative tail companies.
On the topic of tails, I wonder if your distribution would’ve come out differently had the scale been −10, −1, 0, 1, 10.
I can’t quite tell why you think Twitch is bad. It is subject to network effects, kind of a social media company, is that why? And I don’t know what Scale.com is other than some AI company.
Scale’s mission is something like accelerating AI progress, and they have no safety department. So ¯\_(ツ)_/¯ For Twitch I think a bunch of good stuff happens there (chess streamers, Ed Kmett streaming Haskell, or just great gamers), but they’re also in a domain where clickbait and similar Goodharting dynamics are strong, and in the worlds where it gets really big I expect those to dominate.
On the topic of tails, I wonder if your distribution would’ve come out differently had the scale been −10, −1, 0, 1, 10.
I think I would rarely have assigned 10s, due to it being a complex question and this just being a very rough draft.
Another interesting question is whether weighing the rankings by market cap would have made a difference. (But YC didn’t make valuations available in their data, so it would require ~30 min of data entry.)
I wrote up a longer, conceptual review. But I also did a brief data collection, which I’ll post here as others might like to build on or go through a similar exercise.
In 2019 YC released a list of their top 100 portfolio companies ranked by valuation and exit size, where applicable.
So I went through the top 50 companies on this list, and gave each company a ranking ranging from −2 for “Very approval-extracting” to 2 for “Very production-oriented”.
To decide on that number, I asked myself questions like “Would growth of this company seem cancerous?” and “Would I reflectively endorse using this product?”
Companies that scored highly include Doordash, Dropbox and Gusto (all 2′s), and companies that score low include Scale.com (which builds tooling to speed up AI research) and Twitch (-2 and −1).
For comparison, I also did the same exercise with the top 50 S&P500 companies by market cap, with high-scoring ones including Microsoft and Visa, and low-scoring ones including Coca Cola and Salesforce.
This scale is Very Made-up and Maybe Useless. But, if nothing else, it seemed like a useful way to get grounded in some data before thinking further about the post.
Overall, the distributions ended up very similar, though YC did come out with a higher mean, mostly driven by fewer negative tail companies.
Spreadsheet here.
I did the first 20 from each column of your spreadsheet, and got a different result. I hid your answers before writing mine. My rubric was different; instead of focusing on social value, I focused on what type of business relations a company has. You can see my answers here. These are all very noisy, and I’m not entirely confident I didn’t have rating-drift between when I did the YC ones and when I did the S&P ones, but I got a slightly higher score for S&P companies.
In my rubric, things that mean low scores:
You have a Compliance department
A significant portion of your business is oriented towards appeasing gatekeepers (as opposed to there being no relevant gatekeepers, or fighting them not on their normal terms)
The price is not disclosed until you talk to a salesperson
You need your business partners but they don’t need you
Things that mean high scores:
You are creating a new market
Dealing with gatekeepers is not a major concern
Your customers take your price or leave it
You do not operate a call center
Your business partners need you but you don’t need them
We had maximally-different scores (2 vs −2) for Gusto, Microsoft, Facebook, Visa, Mastercard and PayPal. The correlation between our scores was 0.6 for the YC companies, −0.16 for the S&P 500 companies.
Nice, this is interesting!
I don’t understand what this means and what it’s measuring.
This was a great idea, but I think the spreadsheet fails on two fronts—first, it’s measuring the end product rather than the founders and how they operate and attempt to scale, which is the primary thing Benquo is talking about here I believe, and two is that if I ranked these companies I don’t think there would be that much correlation with these rankings.
In the examples from the comment, and judging purely on nature of product since I don’t know the founders or early histories much, I’d have had Twitch as positive while I had Doordash as negative, I’d agree with Dropbox and Gusto, and Scale is a weird case where we think the product is bad if it is real but that’s orthogonal to the main point here.
Looking at the S&P 500 I see the same thing. Amazon at 0 seems insane to me (I’d be +lots) and McDonalds at −2 even more so especially in its early days (The Founder is a very good movie about its origins).
When I read this essay in 2019, I remember getting the impression that approval-extracting vs production-oriented was supposed to be about the behavior of the founders, not the industry the company competes in.
This was a great idea!
I can’t quite tell why you think Twitch is bad. It is subject to network effects, kind of a social media company, is that why? And I don’t know what Scale.com is other than some AI company.
For many of these companies I feel like my opinion changes as they become monopolies. For example, we use Gusto at LW, it’s great. That said, if it became the primary company people used in a country to interact with a part of government, then I could imagine Gusto working with that government to extract money from people in some way. So I like it to a point, then suddenly I might really not like it.
On the topic of tails, I wonder if your distribution would’ve come out differently had the scale been −10, −1, 0, 1, 10.
Scale’s mission is something like accelerating AI progress, and they have no safety department. So ¯\_(ツ)_/¯ For Twitch I think a bunch of good stuff happens there (chess streamers, Ed Kmett streaming Haskell, or just great gamers), but they’re also in a domain where clickbait and similar Goodharting dynamics are strong, and in the worlds where it gets really big I expect those to dominate.
I think I would rarely have assigned 10s, due to it being a complex question and this just being a very rough draft.
Another interesting question is whether weighing the rankings by market cap would have made a difference. (But YC didn’t make valuations available in their data, so it would require ~30 min of data entry.)