On p82 Bostrom argues that even if one research group has access to another’s code, it would still take them months to change approach, which could lead to the leaders getting a full FOOM’s worth of headstart. But I don’t really understand why he thinks this. If I have access to your code, can’t I just run it as soon as I notice you’re running it? True, you could obfuscate it, or protect your production-box more carefully. But if I have that good monitoring systems, I’m probably a government, in which case I can afford to have a team dedicated to reverse-engineering your approach.
Did you find anything this week particularly unpersuasive?
On p82 Bostrom argues that even if one research group has access to another’s code, it would still take them months to change approach, which could lead to the leaders getting a full FOOM’s worth of headstart. But I don’t really understand why he thinks this. If I have access to your code, can’t I just run it as soon as I notice you’re running it? True, you could obfuscate it, or protect your production-box more carefully. But if I have that good monitoring systems, I’m probably a government, in which case I can afford to have a team dedicated to reverse-engineering your approach.
Some reasons that come to mind:
It might take you a while to come to the conclusion that your technology won’t overtake theirs.
You might have slightly different computational resources, and the code might be specific to that.
I don’t see anything about access to code on p82. Are you inferring that from “closely monitor”?
Yes, and good (implicit) point—perhaps Nick had in mind something slightly less close than access to their codebase.