We prefer the world where the project never gets funded in the first place to the world where it gets funded based on lies and then runs 150% of the budget and provides only 50% of the benefit.
Wait, is that true? There are some projects so valuable that I’d want them even at twice the budget and a quarter the incorrectly-estimated value. There are LOTS of projects that I don’t want even if under budget and full estimated value (on the wrong dimensions).
If the project is worthwhile at the real costs and real benefits, then we should be able to get it approved on that basis; if we can’t that indicates another problem. The challenge here is incentivizing a good faith effort to find that information out at all.
Cost estimates are simple lies or errors, and it’s easy (ish) to compare against real costs incurred. Benefits are multidimensional and hard to measure, so are impossible to compare promises to results.
Will a new stadium improve the area’s prestige and economic outlook? How would I even resolve the bet?
I’m a little unclear: if you cannot measure the outcome, how do you know you want the project?
Will a new stadium improve the area’s prestige and economic outlook? How would I even resolve the bet?
Things like land values, the revenues of the businesses around the stadium, and number of permanent jobs working for the stadium and new businesses which rely on stadium traffic are all reasonable kinds of measures. Even if they are difficult to project with accuracy, the same reference class method can be used for the overall impact.
I flatly disagree that it is impossible to compare promises to results; either the things that were promised came to pass, or they did not. This is no different to any other measurement problem. The number of jobs the project will generate has been a popular kind of promise, and both headcount and compensation are routine things economists measure.
If the objection is that deliberately vague promises are impossible to compare, I would agree and that is entirely the fault of the project backers; there’s no way to stop investors in a company from cutting a personal check based on a hand-shake either. However, I note that it is pretty rare for large investments to be made without specific predictions, and I suspect that if projects were put on the same footing as companies people would mostly carry the same assumptions for investing in them.
The ability to compare between different project proposals efficiently seems like an important desiderata to me, and I think it would also have a bearing on the results problem.
Wait, is that true? There are some projects so valuable that I’d want them even at twice the budget and a quarter the incorrectly-estimated value. There are LOTS of projects that I don’t want even if under budget and full estimated value (on the wrong dimensions).
If the project is worthwhile at the real costs and real benefits, then we should be able to get it approved on that basis; if we can’t that indicates another problem. The challenge here is incentivizing a good faith effort to find that information out at all.
Cost estimates are simple lies or errors, and it’s easy (ish) to compare against real costs incurred. Benefits are multidimensional and hard to measure, so are impossible to compare promises to results.
Will a new stadium improve the area’s prestige and economic outlook? How would I even resolve the bet?
I’m a little unclear: if you cannot measure the outcome, how do you know you want the project?
Things like land values, the revenues of the businesses around the stadium, and number of permanent jobs working for the stadium and new businesses which rely on stadium traffic are all reasonable kinds of measures. Even if they are difficult to project with accuracy, the same reference class method can be used for the overall impact.
I flatly disagree that it is impossible to compare promises to results; either the things that were promised came to pass, or they did not. This is no different to any other measurement problem. The number of jobs the project will generate has been a popular kind of promise, and both headcount and compensation are routine things economists measure.
If the objection is that deliberately vague promises are impossible to compare, I would agree and that is entirely the fault of the project backers; there’s no way to stop investors in a company from cutting a personal check based on a hand-shake either. However, I note that it is pretty rare for large investments to be made without specific predictions, and I suspect that if projects were put on the same footing as companies people would mostly carry the same assumptions for investing in them.
The ability to compare between different project proposals efficiently seems like an important desiderata to me, and I think it would also have a bearing on the results problem.