On the infosec thing: ”I simply don’t buy that the infosec for multiple such projects will be anywhere near the infosec of a single project because the overall security ends up being that of the weakest link.” -> nitpick: the important thing isn’t how close the infosec for multiple projects is to the infosec of a single project: it’s how close the infosec for multiple projects is to something like ‘the threshold for good enough infosec, given risk levels and risk tolerance’. That’s obviously very non-trivial to work out -> I agree that a single project would probably have higher infosec than multiple projects (though this doesn’t seem slam dunk to me and I think it does to you) -> concretely, I currently expect that the USG would be able to provide SL4 and maybe SL5 level infosec to 2-5 projects, not just one. Why do you think this isn’t the case?
“Additionally, the more projects there are with a particular capability, the more folk there are who can leak information either by talking or by being spies.” -> It’s not clear to me that a single project would have fewer total people: seems likely that if US AGI development is centralised, it’s part of a big beat China push, and involves throwing a lot of money and people at the problem.
On the infosec thing:
”I simply don’t buy that the infosec for multiple such projects will be anywhere near the infosec of a single project because the overall security ends up being that of the weakest link.”
-> nitpick: the important thing isn’t how close the infosec for multiple projects is to the infosec of a single project: it’s how close the infosec for multiple projects is to something like ‘the threshold for good enough infosec, given risk levels and risk tolerance’. That’s obviously very non-trivial to work out
-> I agree that a single project would probably have higher infosec than multiple projects (though this doesn’t seem slam dunk to me and I think it does to you)
-> concretely, I currently expect that the USG would be able to provide SL4 and maybe SL5 level infosec to 2-5 projects, not just one. Why do you think this isn’t the case?
“Additionally, the more projects there are with a particular capability, the more folk there are who can leak information either by talking or by being spies.”
-> It’s not clear to me that a single project would have fewer total people: seems likely that if US AGI development is centralised, it’s part of a big beat China push, and involves throwing a lot of money and people at the problem.