You probably shouldn’t donate to alignment research. There’s too much useless stuff with too good PR for you to tell what, if anything, is hopeworthy. If you know any young supergenius people who could dedicate their lifeforce to thinking about alignment FROM SCRATCH given some money, consider giving to them.
If there’s some way to fund research that will lead to strong human intelligence amplification, you should do that. I can give some context for that, though not concrete recommendations.
Just to clarify, do you only consider ‘strong human intelligence amplification’ through some internal change, or do you also consider AIs to be part of that? As in, it sounds like you are saying we currently lack the intelligence to make significant progress on alignment research and consider increasing human intelligence to be the best way to make progress. Are you also of the opinion that using AIs to augment alignment researchers and progressively automate alignment research is doomed and not worth consideration? If not, then here.
Not strictly doomed but probably doomed, yeah. You’d have to first do difficult interesting novel philosophy yourself, and then look for things that would have helped with that.
Fair enough. For what it’s worth, I’ve thought a lot about the kind of thing you describe in that comment and partially committing to this direction because I feel like I have enough intuition and insight that those other tools for thought failed to incorporate.
TsviBT didn’t recommend MIRI probably because he receives a paycheck from MIRI and does not want to appear self-serving. I on the other hand have never worked for MIRI and am unlikely ever to (being of the age when people usually retire) so I feel free to recommend MIRI without hesitation or reservation.
MIRI has abandoned hope of anyone’s solving alignment before humanity runs out of time: they continue to employ people with deep expertise in AI alignment, but those employees spend their time explaining why the alignment plans of others will not work.
Most technical alignment researchers are increasing P(doom) because they openly publish results that help both the capability research program and the alignment program, but the alignment program is very unlikely to reach a successful conclusion before the capability program “succeeds”, so publishing the results only shortens the amount of time we have to luck into an effective response or resolution to the AI danger (which again if one appears might not even involve figuring out how to align an AI so that it stays aligned as it becomes an ASI).
There are 2 other (not-for-profit) organizations in the sector that as far as I can tell are probably doing more good than harm, but I don’t know enough about them for it to be a good idea for me to name them here.
I’m no longer employed by MIRI. I think Yudkowsky is by far the best source of technical alignment research insight; but MIRI’s research program was in retrospect probably pretty doomed even before I got there. I can see ways to improve it but I’m not that confident and I can somewhat directly see that I’m probably not capable of carrying out my suggested improvements. And AFAIK, as you say they’re not currently doing very much alignment research. I’m also fine with appearing self-serving; if I were actively doing alignment research, I might recommend myself, though I don’t really think it’s appropriate to do so to a random person who can’t evaluate arguments about alignment research and doesn’t know who to trust. I guess if someone pays me enough I’ll do some alignment research. I recommend myself as one authority among others on strategy regarding strong human intelligence amplification.
I’m not saying that MIRI has some effective plan which more money would help with. I’m only saying that unlike most of the actors accepting money to work in AI Safety, at least they won’t use a donation in a way that makes the situation worse. Specifically, MIRI does not publish insights that help the AI project and is very careful in choosing whom they will teach technical AI skills and knowledge.
at least they won’t use a donation in a way that makes the situation worse
Seems false, they could have problematic effects on discourse if their messaging is poor or seems dumb in retrospect.
I disagree pretty heavily with MIRI which makes this more likely from my perspective.
It seems likely that Yudkowsky has lots of bad effects on discourse right now even from his own lights. I feel pretty good about official MIRI comms activities from my understanding despite a number of disagreements.
Not sure what you’re asking. I think someone trying to work on the technical problem of AI alignment should read Yudkowsky. I think this because… of a whole bunch of the content of ideas and arguments. Would need more context to elaborate, but it doesn’t seem like you’re asking about that.
You probably shouldn’t donate to alignment research. There’s too much useless stuff with too good PR for you to tell what, if anything, is hopeworthy. If you know any young supergenius people who could dedicate their lifeforce to thinking about alignment FROM SCRATCH given some money, consider giving to them.
If there’s some way to fund research that will lead to strong human intelligence amplification, you should do that. I can give some context for that, though not concrete recommendations.
Just to clarify, do you only consider ‘strong human intelligence amplification’ through some internal change, or do you also consider AIs to be part of that? As in, it sounds like you are saying we currently lack the intelligence to make significant progress on alignment research and consider increasing human intelligence to be the best way to make progress. Are you also of the opinion that using AIs to augment alignment researchers and progressively automate alignment research is doomed and not worth consideration? If not, then here.
Not strictly doomed but probably doomed, yeah. You’d have to first do difficult interesting novel philosophy yourself, and then look for things that would have helped with that.
Fair enough. For what it’s worth, I’ve thought a lot about the kind of thing you describe in that comment and partially committing to this direction because I feel like I have enough intuition and insight that those other tools for thought failed to incorporate.
Sure. You can ping me at some point if you want to talk about ideas for that or get my feedback or whatever.
TsviBT didn’t recommend MIRI probably because he receives a paycheck from MIRI and does not want to appear self-serving. I on the other hand have never worked for MIRI and am unlikely ever to (being of the age when people usually retire) so I feel free to recommend MIRI without hesitation or reservation.
MIRI has abandoned hope of anyone’s solving alignment before humanity runs out of time: they continue to employ people with deep expertise in AI alignment, but those employees spend their time explaining why the alignment plans of others will not work.
Most technical alignment researchers are increasing P(doom) because they openly publish results that help both the capability research program and the alignment program, but the alignment program is very unlikely to reach a successful conclusion before the capability program “succeeds”, so publishing the results only shortens the amount of time we have to luck into an effective response or resolution to the AI danger (which again if one appears might not even involve figuring out how to align an AI so that it stays aligned as it becomes an ASI).
There are 2 other (not-for-profit) organizations in the sector that as far as I can tell are probably doing more good than harm, but I don’t know enough about them for it to be a good idea for me to name them here.
I’m no longer employed by MIRI. I think Yudkowsky is by far the best source of technical alignment research insight; but MIRI’s research program was in retrospect probably pretty doomed even before I got there. I can see ways to improve it but I’m not that confident and I can somewhat directly see that I’m probably not capable of carrying out my suggested improvements. And AFAIK, as you say they’re not currently doing very much alignment research. I’m also fine with appearing self-serving; if I were actively doing alignment research, I might recommend myself, though I don’t really think it’s appropriate to do so to a random person who can’t evaluate arguments about alignment research and doesn’t know who to trust. I guess if someone pays me enough I’ll do some alignment research. I recommend myself as one authority among others on strategy regarding strong human intelligence amplification.
I’m not saying that MIRI has some effective plan which more money would help with. I’m only saying that unlike most of the actors accepting money to work in AI Safety, at least they won’t use a donation in a way that makes the situation worse. Specifically, MIRI does not publish insights that help the AI project and is very careful in choosing whom they will teach technical AI skills and knowledge.
Seems false, they could have problematic effects on discourse if their messaging is poor or seems dumb in retrospect.
I disagree pretty heavily with MIRI which makes this more likely from my perspective.
It seems likely that Yudkowsky has lots of bad effects on discourse right now even from his own lights. I feel pretty good about official MIRI comms activities from my understanding despite a number of disagreements.
Not sure what you’re asking. I think someone trying to work on the technical problem of AI alignment should read Yudkowsky. I think this because… of a whole bunch of the content of ideas and arguments. Would need more context to elaborate, but it doesn’t seem like you’re asking about that.
I still don’t know what you mean.
please do!
https://www.lesswrong.com/posts/jTiSWHKAtnyA723LE/overview-of-strong-human-intelligence-amplification-methods
DM’d
Would you mind posting that information here? I am also interested (as, I’m sure, are others).
https://www.lesswrong.com/posts/jTiSWHKAtnyA723LE/overview-of-strong-human-intelligence-amplification-methods