I enjoyed this really a lot, and while I don’t have anything insightful to add, I gave five bucks to MIRI to encourage more of this sort of thing.
(By “this sort of thing” I mean detailed descriptions of the actual problems you are working on as regards FAI research. I gather that you consider a lot of it too dangerous to describe in public, but then I don’t get to enjoy reading about it. So I would like to encourage you sharing some of the fun problems sometimes. This one was fun.)
Not ‘a lot’ and present-day non-sharing imperatives are driven by an (obvious) strategy to accumulate a long-term advantage for FAI projects over AGI projects which is impossible if all lines of research are shared at all points when they are not yet imminently dangerous. No present-day knowledge is imminently dangerous AFAIK.
present-day non-sharing imperatives are driven by an (obvious) strategy to accumulate a long-term advantage for FAI projects over AGI projects
Do you believe this to be possible? In modern times with high mobility of information and people I have strong doubts a gnostic approach would work. You can hide small, specific, contained “trade secrets”, you can’t hide a large body of knowledge that needs to be actively developed.
I can’t help but remember HPJEV talk about plausible deniability and how that relates to you telling people whether there is dangerous knowledge out there.
I thought this was an engaging, well-written summary targeted to the general audience, and I’d like to encourage more articles along these lines. So as a follow-up question: How much income for MIRI would it take, per article, for the beneficial effects of sharing non-dangerous research to outweigh the negatives?
(Gah, the editor in me WINCES at that sentence. Is it clear enough or should I re-write? I’m asking how much I-slash-we should kick in per article to make the whole thing generally worth your while.)
Given how many underpaid science writers are out there, I’d have to say that ~50k/year would probably do it for a pretty good one, especially given the ‘good cause’ bonus to happiness that any qualified individual would understand and value. But is even 1k/week in donations realistic? What are the page view numbers? I’d pay $5 for a good article on a valuable topic; how many others would as well? I suspect the numbers don’t add up, but I don’t even have an order-of-magnitude estimate on current or potential readers, so I can’t myself say.
You need not only a good science writer, but one who either already groks the problem, or can be made to do so with a quick explanation.
Furthermore, they need to have the above qualifications without being capable of doing primary research on the problem (this is the issue with Eliezer—he would certainly be capable of doing it, but his time is better spent elsewhere.)
Well, $100K/year would probably pay someone to write things up full time, if we only had the right candidate hire for it—I’m not sure we do. The issue is almost never danger, it’s just that writing stuff up is hard.
I can certainly see that people who can both understand these issues and write them up for a general audience would be rare. Working in your favor is the fact that writers in general are terribly underpaid, and a lot of smart tech journalists have been laid off in recent years. (I used to be the news editor for Dr. Dobb’s Journal, and although I am not looking for a job right now, I have contacts who could probably fill the position for you.)
But I did some back-of-the-envelope calculations and it doesn’t seem like this effort would pay for itself. I doubt you have enough questions like this to cover a daily article, and for a weekly one you’d need to take in over $2K in donations (counting taxes) to cover your writer’s salary. And that seems...unlikely.
I enjoyed this really a lot, and while I don’t have anything insightful to add, I gave five bucks to MIRI to encourage more of this sort of thing.
(By “this sort of thing” I mean detailed descriptions of the actual problems you are working on as regards FAI research. I gather that you consider a lot of it too dangerous to describe in public, but then I don’t get to enjoy reading about it. So I would like to encourage you sharing some of the fun problems sometimes. This one was fun.)
Not ‘a lot’ and present-day non-sharing imperatives are driven by an (obvious) strategy to accumulate a long-term advantage for FAI projects over AGI projects which is impossible if all lines of research are shared at all points when they are not yet imminently dangerous. No present-day knowledge is imminently dangerous AFAIK.
Do you believe this to be possible? In modern times with high mobility of information and people I have strong doubts a gnostic approach would work. You can hide small, specific, contained “trade secrets”, you can’t hide a large body of knowledge that needs to be actively developed.
I can’t help but remember HPJEV talk about plausible deniability and how that relates to you telling people whether there is dangerous knowledge out there.
Thanks for the clarification!
I thought this was an engaging, well-written summary targeted to the general audience, and I’d like to encourage more articles along these lines. So as a follow-up question: How much income for MIRI would it take, per article, for the beneficial effects of sharing non-dangerous research to outweigh the negatives?
(Gah, the editor in me WINCES at that sentence. Is it clear enough or should I re-write? I’m asking how much I-slash-we should kick in per article to make the whole thing generally worth your while.)
Given how many underpaid science writers are out there, I’d have to say that ~50k/year would probably do it for a pretty good one, especially given the ‘good cause’ bonus to happiness that any qualified individual would understand and value. But is even 1k/week in donations realistic? What are the page view numbers? I’d pay $5 for a good article on a valuable topic; how many others would as well? I suspect the numbers don’t add up, but I don’t even have an order-of-magnitude estimate on current or potential readers, so I can’t myself say.
You need not only a good science writer, but one who either already groks the problem, or can be made to do so with a quick explanation.
Furthermore, they need to have the above qualifications without being capable of doing primary research on the problem (this is the issue with Eliezer—he would certainly be capable of doing it, but his time is better spent elsewhere.)
Well, $100K/year would probably pay someone to write things up full time, if we only had the right candidate hire for it—I’m not sure we do. The issue is almost never danger, it’s just that writing stuff up is hard.
Apropos the above conversation: Do you know Annalee Newitz? (Of io9). If not, would you like to? I think you guys would get on like a house on fire.
I can certainly see that people who can both understand these issues and write them up for a general audience would be rare. Working in your favor is the fact that writers in general are terribly underpaid, and a lot of smart tech journalists have been laid off in recent years. (I used to be the news editor for Dr. Dobb’s Journal, and although I am not looking for a job right now, I have contacts who could probably fill the position for you.)
But I did some back-of-the-envelope calculations and it doesn’t seem like this effort would pay for itself. I doubt you have enough questions like this to cover a daily article, and for a weekly one you’d need to take in over $2K in donations (counting taxes) to cover your writer’s salary. And that seems...unlikely.
Sad! But I get it.