“And if we cannot make progress on Friendly AI because we are not prepared, this does not mean we do not need Friendly AI.”
Friendly AI will be the only thing we need, and unfriendly AI will be the worst possible thing that could happen. If the problem seems impossible to solve, that doesn’t matter because this isn’t about diminishing returns or ROI, it’s about humanity’s final chance at figuring things out. The payoff of a single working solution will be a payoff that continues yielding dividends for billions of years.
“And if we cannot make progress on Friendly AI because we are not prepared, this does not mean we do not need Friendly AI.”
Friendly AI will be the only thing we need, and unfriendly AI will be the worst possible thing that could happen. If the problem seems impossible to solve, that doesn’t matter because this isn’t about diminishing returns or ROI, it’s about humanity’s final chance at figuring things out. The payoff of a single working solution will be a payoff that continues yielding dividends for billions of years.