My model of FAI development says that you have to get most of the way to being able to build an AGI just to be able to start working on many Friendliness-specific problems, and solving those problems would take a long time relative to finishing rest of the AGI capability work.
Agree, though luckily there are other Friendliness-specific problems that we can start solving right now.
Unless you’re flying completely below the radar, which is incompatible with your plan for funding via public donations, what is stopping your unpublished results from being stolen or leaked in the mean time?
Presumably, security technology similar to what has mostly worked for the Manhattan project, secret NSA projects, etc. But yeah, it’s a big worry. But what did you have in mind about flying completely under the radar? There are versions of an FAI team that could be funded pretty discretely by just one person.
Agree, though luckily there are other Friendliness-specific problems that we can start solving right now.
I listed some in another comment, but they are not the current focus of MIRI research. Instead, MIRI is focusing on FAI-relevant problems that do shorten AI timelines (i.e., working on “get most of the way to being able to build an AGI”), such as decision theory and logical uncertainty.
Presumably, security technology similar to what has mostly worked for the Manhattan project, secret NSA projects, etc.
As I noted in previous comments, the economics of information security seems to greatly favor the offense, so you have to have to spend much more resources than your attackers in order to maintain secrets.
But what did you have in mind about flying completely under the radar? There are versions of an FAI team that could be funded pretty discretely by just one person.
That’s probably the best bet as far as avoiding having your results stolen, but introduces other problems, such as how to attract talent, and whether you can fund a large enough team that way. (Small teams might increase the chances of accidental UFAI creation, since there would be less people to look out for errors.) And given that Eliezer is probably already on the radar of most AGI researchers, you’d have to find a replacement for him on this “under the radar” team.
Agree, though luckily there are other Friendliness-specific problems that we can start solving right now.
Presumably, security technology similar to what has mostly worked for the Manhattan project, secret NSA projects, etc. But yeah, it’s a big worry. But what did you have in mind about flying completely under the radar? There are versions of an FAI team that could be funded pretty discretely by just one person.
I listed some in another comment, but they are not the current focus of MIRI research. Instead, MIRI is focusing on FAI-relevant problems that do shorten AI timelines (i.e., working on “get most of the way to being able to build an AGI”), such as decision theory and logical uncertainty.
As I noted in previous comments, the economics of information security seems to greatly favor the offense, so you have to have to spend much more resources than your attackers in order to maintain secrets.
That’s probably the best bet as far as avoiding having your results stolen, but introduces other problems, such as how to attract talent, and whether you can fund a large enough team that way. (Small teams might increase the chances of accidental UFAI creation, since there would be less people to look out for errors.) And given that Eliezer is probably already on the radar of most AGI researchers, you’d have to find a replacement for him on this “under the radar” team.