I do agree that equilibrium dynamics mean that the failure rate is not fixed, and that this is underappreciated.
But, if the investors judge that the payoff of AI safety lab success would be above average, they should be willing to invest more and accept above average failures rates too. Failure rate increases with additional investment, and investment should increase until expected returns even out across options for further investment.
The harder a problem is, the higher the expected failure rate at any given rate of investment at any given point in time, and the investment won’t happen until and unless the expected return makes it pencil out.
Side note: As I understand it, the real equilibrium ROI for capital in general has been 4-5% on average for as far back as we have records. I find this suspicious, as I have seen no good studies on why. But my headcanon has been that this gives a doubling time about equal to the human generation time. AKA parents and societies save until they’ve saved enough to hand their wealth, undiluted, down to their children, on average.
Right yeah, that’s exactly it.
And we agree on that, yes.
I do agree that equilibrium dynamics mean that the failure rate is not fixed, and that this is underappreciated.
But, if the investors judge that the payoff of AI safety lab success would be above average, they should be willing to invest more and accept above average failures rates too. Failure rate increases with additional investment, and investment should increase until expected returns even out across options for further investment.
The harder a problem is, the higher the expected failure rate at any given rate of investment at any given point in time, and the investment won’t happen until and unless the expected return makes it pencil out.
Side note: As I understand it, the real equilibrium ROI for capital in general has been 4-5% on average for as far back as we have records. I find this suspicious, as I have seen no good studies on why. But my headcanon has been that this gives a doubling time about equal to the human generation time. AKA parents and societies save until they’ve saved enough to hand their wealth, undiluted, down to their children, on average.