Sometimes I see people use the low-info heuristic as a “baseline” and then apply some sort of “fudge factor” for the illegible information that isn’t incorporated into the baseline—something like “the baseline probability of this startup succeeding is 10%, but the founders seem really determined so I’ll guesstimate that gives them a 50% higher probability of success.” In principle I could imagine this working reasonably well, but in practice most people who do this aren’t willing to apply as large of a fudge factor as appropriate.
The last company I worked for was a tech scouting, market research, and consulting firm, and a big part of what they do is profile start-ups, using a standard format and scorecard, based on a 1 hour interview + background knowledge of an industry. One time they bought a data science company and turned them loose on a decade of profiles, and found several results like “hey, if this score is a 4⁄5 or 5⁄5 then the company is 2x or 4x more likely to have a successful exit, respectively.” They put this in a white paper, sent it out to clients, and then… nothing. Never used it for marketing, sales, internal research process improvement. It always seemed bizarre to me, that “Hey, we know our process can quadruple your odds of finding startups that will succeed,” when that was our whole job, just… didn’t seem to motivate the people in charge.
In any case, my point is, it is very easy to find subsets of companies that outperform the 90% failure figure if that is what you optimize for, and if what you hear isn’t only filtered through the way the startups frame their pitches to investors.
The last company I worked for was a tech scouting, market research, and consulting firm, and a big part of what they do is profile start-ups, using a standard format and scorecard, based on a 1 hour interview + background knowledge of an industry. One time they bought a data science company and turned them loose on a decade of profiles, and found several results like “hey, if this score is a 4⁄5 or 5⁄5 then the company is 2x or 4x more likely to have a successful exit, respectively.” They put this in a white paper, sent it out to clients, and then… nothing. Never used it for marketing, sales, internal research process improvement. It always seemed bizarre to me, that “Hey, we know our process can quadruple your odds of finding startups that will succeed,” when that was our whole job, just… didn’t seem to motivate the people in charge.
In any case, my point is, it is very easy to find subsets of companies that outperform the 90% failure figure if that is what you optimize for, and if what you hear isn’t only filtered through the way the startups frame their pitches to investors.