I was mainly talking about the current margin when I talked about how excited I am about the theoretical vs empirical work I see “going on” right now and how excited I tend to be about currently-active researchers who are doing theory vs empirical research. And I was talking about the future when I said that I expect empirical work to end up with the lion’s share of credit for AI risk reduction.
Eliezer, Bostrom, and co certainly made a big impact in raising the problem to people’s awareness and articulating some of its contours. It’s kind of a matter of semantics whether you want to call that “theoretical research” or “problem advocacy” / “cause prioritization” / “community building” / whatever, and no matter which bucket you put it in I agree it’ll probably end up with an outsized impact for x-risk-reduction, by bringing the problem to attention sooner than it would have otherwise been brought to attention and therefore probably allowing more work to happen on it before TAI is developed.
But just like how founding CEOs tend to end up with ~10% equity once their companies have grown large, I don’t think this historical problem-advocacy-slash-theoretical-research work alone will end up with a very large amount of total credit.
On the main thrust of my point, I’m significantly less excited about MIRI-sphere work that is much less like “articulating a problem and advocating for its importance” and much more like “attempting to solve a problem.” E.g. stuff like logical inductors, embedded agency, etc seem a lot less valuable to me than stuff like the orthogonality thesis and so on.
I was mainly talking about the current margin when I talked about how excited I am about the theoretical vs empirical work I see “going on” right now and how excited I tend to be about currently-active researchers who are doing theory vs empirical research. And I was talking about the future when I said that I expect empirical work to end up with the lion’s share of credit for AI risk reduction.
Eliezer, Bostrom, and co certainly made a big impact in raising the problem to people’s awareness and articulating some of its contours. It’s kind of a matter of semantics whether you want to call that “theoretical research” or “problem advocacy” / “cause prioritization” / “community building” / whatever, and no matter which bucket you put it in I agree it’ll probably end up with an outsized impact for x-risk-reduction, by bringing the problem to attention sooner than it would have otherwise been brought to attention and therefore probably allowing more work to happen on it before TAI is developed.
But just like how founding CEOs tend to end up with ~10% equity once their companies have grown large, I don’t think this historical problem-advocacy-slash-theoretical-research work alone will end up with a very large amount of total credit.
On the main thrust of my point, I’m significantly less excited about MIRI-sphere work that is much less like “articulating a problem and advocating for its importance” and much more like “attempting to solve a problem.” E.g. stuff like logical inductors, embedded agency, etc seem a lot less valuable to me than stuff like the orthogonality thesis and so on.