Note that you’re wrongly discouraging people from doing strategy research by saying that they need to catch up to insiders’ unpublished knowledge when they really don’t.
What makes you say that? I believe you can reinvent much of what Eliezer and Carl and Bostrom and a few others already know but haven’t written down. Not sure that’s true for almost most everyone else.
I read the idea as being that people rediscovering and writing up stuff that goes 5% towards what E/C/N have already figured out but haven’t written down would be a net positive and it’s a bad idea to discourage this. It seems like there’s something to that, to the degree that getting the existing stuff written up isn’t an available option—increasing the level of publicly available strategic research could be useful even if the vast majority of it doesn’t advance the state of the art, if it leads to many more people vetting it in the long run. I do think there is probably a tradeoff, where Eliezer &c might not be motivated to comment on other people’s posts all that much, making it difficult to see what is the current state of the art and what are ideas that the poster just hasn’t figured out the straight-forward counter-arguments to. I don’t know how to deal with that, but encouraging discussion that is high quality compared to currently publicly available strategy work still seems quite likely to be a net positive?
One way to accelerate the production of strategy exposition is to lower one’s standards. It’s much easier to sketch one’s quick thoughts on an issue than it is to write a well-organized, clearly-expressed, well-referenced, reader-tested analysis (like When Will AI Be Created?), and this is often enough to provoke some productive debate (at least on Less Wrong). See e.g. Reply to Holden on Tool AI and Do Earths with slower economic growth have a better chance at FAI?.
So, in the next few days I’ll post my “quick and dirty” thoughts on one strategic issue (IA and FAI) to LW Discussion, and see what comes of it.
Glad to hear that & looking forward to seeing how it works! I very much understand that one might be concerned about posting “quick and dirty” thoughts (I find it so very difficult to lower my own standards even when it’s obviously blocking me from getting stuff done), but there seems to be little cost of trying it with a Discussion post and seeing how it goes—yay value of information! :-)
I read the idea as being that people rediscovering and writing up stuff that goes 5% towards what E/C/N have already figured out but haven’t written down would be a net positive and it’s a bad idea to discourage this. It seems like there’s something to that, to the degree that getting the existing stuff written up isn’t an available option—increasing the level of publicly available strategic research could be useful even if the vast majority of it doesn’t advance the state of the art, if it leads to many more people vetting it in the long run. I do think there is probably a tradeoff, where Eliezer &c might not be motivated to comment on other people’s posts all that much, making it difficult to see what is the current state of the art and what are ideas that the poster just hasn’t figured out the straight-forward counter-arguments to. I don’t know how to deal with that, but encouraging discussion that is high quality compared to currently publicly available strategy work still seems quite likely to be a net positive?
One way to accelerate the production of strategy exposition is to lower one’s standards. It’s much easier to sketch one’s quick thoughts on an issue than it is to write a well-organized, clearly-expressed, well-referenced, reader-tested analysis (like When Will AI Be Created?), and this is often enough to provoke some productive debate (at least on Less Wrong). See e.g. Reply to Holden on Tool AI and Do Earths with slower economic growth have a better chance at FAI?.
So, in the next few days I’ll post my “quick and dirty” thoughts on one strategic issue (IA and FAI) to LW Discussion, and see what comes of it.
Glad to hear that & looking forward to seeing how it works! I very much understand that one might be concerned about posting “quick and dirty” thoughts (I find it so very difficult to lower my own standards even when it’s obviously blocking me from getting stuff done), but there seems to be little cost of trying it with a Discussion post and seeing how it goes—yay value of information! :-)
The experiment seems to have failed.
Drats. But also, yay, information! Thanks for trying this!
ETA: Worth noting that I found that post useful, though.