Qin Shi Huang sought to live forever. So, he recruited a multitude of advisors to find the hidden secret to immortality. Unfortunately, the men he recruited had no ability to assess which magicians or alchemists might be experts in immortality elixirs. Furthermore, the emperor had no ability to assess which advisors might be experts in assessing experts in immortality elixirs. (Note even further that there are no true experts in the relevant domains here. This may remind you of fields such as technological forecasting.) In the end, as these things tended to go, a lot of advisors got executed.
...
In domains where reality does not give good feedback, they need to have a set of well-honed heuristics or proxy feedback methods to correct for better output if the result is going to be reliably good (this goes for, e.g., philosophy, sociology, long-term prediction). In domains where reality can give good feedback, they don’t necessarily need well-honed heuristics or proxy feedback methods (e.g., massage, auto repair, swordfighting, etc.). All else equal, superior feedback loops have the following attributes (idealized versions below):
Speed (you learn about discrepancies between current and desired output quickly after taking an action so you can course-correct)
Frequency (the feedback loop happens frequently, giving you more samples to calibrate on)
Validity (the feedback loop is helping you get closer to the output you actually care about)
Reliability (the feedback loop consistently returns similar discrepancies in response to you taking similar actions)
Detail (the feedback loop gives you a large amount of information about the difference between current and desired output)
Saliency (the feedback loop delivers attentionally or motivationally salient feedback)
...
-http://effective-altruism.com/ea/tk/expertise_assessment/