Given that systems of software which learn can eventually bring about ‘transformative’ impact (defined as ‘impact comparable to the industrial revolution’), the most important thing to work on is AI. Given that the open problems in learning software between now and its transformativity can be solved in a multitude of ways, some of those solutions will be more or less beneficial, less or more dangerous, meaning there’s a lever that altruistic researchers can use to steer outcomes in these open problems. Given the difficulty of social dilemmas and coordination, we need research that is aimed at improving single-multi, multi-single, and multi-multi capabilities until those capabilities outpace single-single capabilities. Given the increase in economic and military power implied by transformative systems, civilization could be irrevocably damaged by simple coordination failures.
Given that systems of software which learn can eventually bring about ‘transformative’ impact (defined as ‘impact comparable to the industrial revolution’), the most important thing to work on is AI. Given that the open problems in learning software between now and its transformativity can be solved in a multitude of ways, some of those solutions will be more or less beneficial, less or more dangerous, meaning there’s a lever that altruistic researchers can use to steer outcomes in these open problems. Given the difficulty of social dilemmas and coordination, we need research that is aimed at improving single-multi, multi-single, and multi-multi capabilities until those capabilities outpace single-single capabilities. Given the increase in economic and military power implied by transformative systems, civilization could be irrevocably damaged by simple coordination failures.