SPRs can be gamed much more directly than human experts. For example, imagine an SPR in place of all hiring managers. In our current place, with hiring managers, we can guess at what goes in to their decisionmaking and attempt to optimize for it, but because each manager is somewhat different, we can’t know that well. A single SPR that took over for all the managers, or even a couple of very popular ones, would strongly encourage applicants to optimize for the variable most weighted in the equation. Over time this would likely decrease the value of the SPR back to that of a human expert.
This has a name in the literature, but I can’t remember it at the moment. You see this problem in, for example, the current obsessive focus on GDP as the only measure of national well-being. Now that we’ve had that measure for some time, we’re able to have countries whose GDP is improving but who suck on lots of other measures, and thus politicians who are proud of what they’ve done but who are hated by the people.
Yes, in some cases, this would cause us to improve the SPR to the point where it accurately reflected the qualities that go into success. But that’s not a proven thing.
That said, I’d really like to see a wiki or other attempting-to-be-complete resource for finding an SPR for any particular application. Anyone got one?
me! me! I’ll be there! I’ve wanted a meetup here for a long time, but was pretty sure nobody was here.