The difference is in size of economic output. “Do science” in a sense “produce scientifically-looking output” can even modern GPT-4, but “do science” in sense “find novel (surprising for domain experts) economically valuable discoveries” is totally different thing.
“Design fusion rocket” can have multiple levels of quality. If you mean by this “output instruction using which team of moderately competent engineers can launch fusion rocket on first try”, I think, corresponding cognitive engine has all elements necessary to make in generically dangerous.
The difference is in size of economic output. “Do science” in a sense “produce scientifically-looking output” can even modern GPT-4, but “do science” in sense “find novel (surprising for domain experts) economically valuable discoveries” is totally different thing. “Design fusion rocket” can have multiple levels of quality. If you mean by this “output instruction using which team of moderately competent engineers can launch fusion rocket on first try”, I think, corresponding cognitive engine has all elements necessary to make in generically dangerous.