What is the value of strategy vs. object-level progress toward a positive Singularity?
I think strategy is currently being undervalued, although this workshop and other recent SIAI work is alleviating that concern quite a bit. I’m still not sure why there is so little discussion of Singularity strategies on LW, compared to say decision theory, CEV, or Oracle AI, even as SIAI has started doing a lot more work in that area. Is it that most LW posts can be considered replies/followups to Eliezer’s posts, and he never wrote one about Singularity strategies?
Almost none of the strategy work has been published, but we’ve had thousands upon thousands of hours of dialogue between SIAI and FHI researchers and others like Paul Christiano (who is now a research associate) and Owain Evans. Carl has been in Oxford the last two weeks working with FHI folk on strategy issues almost every day. SIAI had a series of 10-hour strategy meetings in January. Believe me, lots of strategy work is being done, but I understand that you’re not seeing hardly any of it. Much of the strategy work that’s been done by SIAI and FHI in the past five years is, thankfully, ending up in Bostrom’s forthcoming monograph on machine superintelligence.
I think strategy is currently being undervalued, although this workshop and other recent SIAI work is alleviating that concern quite a bit. I’m still not sure why there is so little discussion of Singularity strategies on LW, compared to say decision theory, CEV, or Oracle AI, even as SIAI has started doing a lot more work in that area. Is it that most LW posts can be considered replies/followups to Eliezer’s posts, and he never wrote one about Singularity strategies?
Almost none of the strategy work has been published, but we’ve had thousands upon thousands of hours of dialogue between SIAI and FHI researchers and others like Paul Christiano (who is now a research associate) and Owain Evans. Carl has been in Oxford the last two weeks working with FHI folk on strategy issues almost every day. SIAI had a series of 10-hour strategy meetings in January. Believe me, lots of strategy work is being done, but I understand that you’re not seeing hardly any of it. Much of the strategy work that’s been done by SIAI and FHI in the past five years is, thankfully, ending up in Bostrom’s forthcoming monograph on machine superintelligence.