This is one of my concerns about the SIAI. They seem to be enthusiastic about caution—but excessive caution in this area seems likely to increase the chances of an undesirable outcome—via the mechanism in the link—so they may be having a particularly negative impact.
It is challenging to know what will help:
Maybe pointing at machine intelligence and shouting “DANGER!” and “WEAPON!” will just attract the attention of the military.
Maybe getting the safety-conscious teams to slow down will mean a greater chance of the unscrupulous teams getting there first.
This is one of my concerns about the SIAI. They seem to be enthusiastic about caution—but excessive caution in this area seems likely to increase the chances of an undesirable outcome—via the mechanism in the link—so they may be having a particularly negative impact.