How to you measure progress when finding out that you’ve made a mistake and need to dump a bunch of the work you’ve done is likely to be an important part of the task?
Good question. What do you think of how Givewell does it? (Because they do assess their own performance in accord with their overall emphasis on transparency and metrics, and they are also in the research business, so that they, like us, often need to backtrack and re-assess.)
I like the piece from Givewell, but they’re doing things which are much easier to measure.
My impression is that SIAI is at a stage where most of what can be measured is inputs (money raised, hours worked) rather than outputs, and it’s hard to tell whether an output (a new piece of theory, for example) is actually getting closer to one’s goals.
I’m not saying that SIAI’s work is unimportant, but evaluating it may be more a matter of logic than measurement.
How to you measure progress when finding out that you’ve made a mistake and need to dump a bunch of the work you’ve done is likely to be an important part of the task?
Good question. What do you think of how Givewell does it? (Because they do assess their own performance in accord with their overall emphasis on transparency and metrics, and they are also in the research business, so that they, like us, often need to backtrack and re-assess.)
I like the piece from Givewell, but they’re doing things which are much easier to measure.
My impression is that SIAI is at a stage where most of what can be measured is inputs (money raised, hours worked) rather than outputs, and it’s hard to tell whether an output (a new piece of theory, for example) is actually getting closer to one’s goals.
I’m not saying that SIAI’s work is unimportant, but evaluating it may be more a matter of logic than measurement.