At the core, this is a reminder to not publish things that will help more with capabilities than alignment. That’s perfectly reasonable.
The tone of the post suggests erring on the side of “safety” by not publishing things that have an uncertain safety/capabilities balance. I hope that wasn’t the intent.
Because that does not make sense. Anything that advances alignment more than safety in expectation should be published.
You have to make a difficult judgment call for each publication. Be mindful of your bias in wanting to publish to show off your work and ideas. Get others’ insights if you can do so reasonably quickly.
But at the end of the day, you have to make that judgment call. There’s no consolation prize for saying “at least I didn’t make the world end faster”. If you’re a utilitarian, winning the future is the only goal.
(If you’re not a utilitarian, you might actually want a resolution faster so you and your loved ones have higher odds of surviving into the far future.)
to make people weigh whether to publish or not, because I think some people are not weighing this enough
to give some arguments in favor of “you might be systematically overestimating the utility of publishing”, because I think some people are doing that
I agree people should take the utilitalianly optimal action, I just think they’re doing the utilitarian calculus wrong or not doing the calculus at all.
At the core, this is a reminder to not publish things that will help more with capabilities than alignment. That’s perfectly reasonable.
The tone of the post suggests erring on the side of “safety” by not publishing things that have an uncertain safety/capabilities balance. I hope that wasn’t the intent.
Because that does not make sense. Anything that advances alignment more than safety in expectation should be published.
You have to make a difficult judgment call for each publication. Be mindful of your bias in wanting to publish to show off your work and ideas. Get others’ insights if you can do so reasonably quickly.
But at the end of the day, you have to make that judgment call. There’s no consolation prize for saying “at least I didn’t make the world end faster”. If you’re a utilitarian, winning the future is the only goal.
(If you’re not a utilitarian, you might actually want a resolution faster so you and your loved ones have higher odds of surviving into the far future.)
I am a utilitarian and agree with your comment.
The intent of the post was
to make people weigh whether to publish or not, because I think some people are not weighing this enough
to give some arguments in favor of “you might be systematically overestimating the utility of publishing”, because I think some people are doing that
I agree people should take the utilitalianly optimal action, I just think they’re doing the utilitarian calculus wrong or not doing the calculus at all.