This is a difficult problem whose implications go well beyond evaluating charities. Many people seem to defer their evaluation of experts to the experts, but then you have to figure out how to qualify those experts, and I haven’t yet seen a good solution to that.
Some heuristics that I use instead:
—Does the expert produce powerful, visible effects in their domain of expertise which non-experts can’t duplicate? If so, they’re probably reliable within their domain. (For example, engineers can build bridges and authors can make compelling stories, so they’re probably reliable in those fields.) This is only useful in cases where a non-expert can evaluate the product’s quality; it won’t help a non-mathematician evaluate theoretical physics. —Are the domain experts split into many disagreeing camps? If so, at most one camp is right, which means most of the experts are wrong, and the field isn’t reliable. (So this rules out, e.g., experts on nutrition.) This one is a tools for assessing domains of expertise, and won’t tell you much about individual experts.
This is a difficult problem whose implications go well beyond evaluating charities. Many people seem to defer their evaluation of experts to the experts, but then you have to figure out how to qualify those experts, and I haven’t yet seen a good solution to that.
Some heuristics that I use instead:
—Does the expert produce powerful, visible effects in their domain of expertise which non-experts can’t duplicate? If so, they’re probably reliable within their domain. (For example, engineers can build bridges and authors can make compelling stories, so they’re probably reliable in those fields.) This is only useful in cases where a non-expert can evaluate the product’s quality; it won’t help a non-mathematician evaluate theoretical physics.
—Are the domain experts split into many disagreeing camps? If so, at most one camp is right, which means most of the experts are wrong, and the field isn’t reliable. (So this rules out, e.g., experts on nutrition.) This one is a tools for assessing domains of expertise, and won’t tell you much about individual experts.