What if it’s hard to tell, in advance, who is “the right sort of people”, and who is “the wrong sort”?
Well, perhaps that’s not so bad, if the decision to include is not irrevocable; if, having included someone, you can easily exclude them later.
But what if it’s not easy to exclude them later?
If you depend on having only the right sort of person within your system, then might a wrong sort of person, included even temporarily, do serious damage?
What if removing someone is very costly?
All of these are only the risks associated with false positives. What about false negatives?
What if it’s hard to identify the right sort of person, and such people are so rare (and/or you need so many of them) that only including those you’ve identified as the right sort means that you miss out on many people whose inclusion would substantially increase your system’s value, and without whom you risk failure?
Also, are there costs imposed by your selection procedure itself? (These costs might be borne by the system and those within it, by prospective members, or by uninvolved others—the latter sort usually being known as “externalities”.)
Such costs may actually be (part of) the selection mechanism—but they aren’t always.
Are there consequences of the use of selective methods, that shape your system in ways other than the simple fact of the methods’ results?
Missing what I’d consider the biggest problem: it seems like the vast majority of problems in real-world social systems do not stem from malign or unusually incompetent actors; they stem from failures of coordination, failures of information-passing, failures of anyone with the freedom to act noticing that nobody is performing some crucial role, and other primarily-structural issues. Insofar as that’s true, selection basically cannot solve the majority of problems in social systems.
Conversely, well-designed structures can solve selection failures, at least to a much larger extent than selection can solve structural failures. Designing systems to work fine with mostly-average people and be robust to a few negative outliers is difficult, but possible. So there’s a real asymmetry between structural vs selective approaches.
Problems with selective methods:
What if it’s hard to tell, in advance, who is “the right sort of people”, and who is “the wrong sort”?
Well, perhaps that’s not so bad, if the decision to include is not irrevocable; if, having included someone, you can easily exclude them later.
But what if it’s not easy to exclude them later?
If you depend on having only the right sort of person within your system, then might a wrong sort of person, included even temporarily, do serious damage?
What if removing someone is very costly?
All of these are only the risks associated with false positives. What about false negatives?
What if it’s hard to identify the right sort of person, and such people are so rare (and/or you need so many of them) that only including those you’ve identified as the right sort means that you miss out on many people whose inclusion would substantially increase your system’s value, and without whom you risk failure?
Also, are there costs imposed by your selection procedure itself? (These costs might be borne by the system and those within it, by prospective members, or by uninvolved others—the latter sort usually being known as “externalities”.)
Such costs may actually be (part of) the selection mechanism—but they aren’t always.
Are there consequences of the use of selective methods, that shape your system in ways other than the simple fact of the methods’ results?
Missing what I’d consider the biggest problem: it seems like the vast majority of problems in real-world social systems do not stem from malign or unusually incompetent actors; they stem from failures of coordination, failures of information-passing, failures of anyone with the freedom to act noticing that nobody is performing some crucial role, and other primarily-structural issues. Insofar as that’s true, selection basically cannot solve the majority of problems in social systems.
Conversely, well-designed structures can solve selection failures, at least to a much larger extent than selection can solve structural failures. Designing systems to work fine with mostly-average people and be robust to a few negative outliers is difficult, but possible. So there’s a real asymmetry between structural vs selective approaches.