First, some disclosure :) My research is funded by MIRI. On the other hand, all of my opinions are my own and do not represent MIRI or anyone else associated with MIRI.
The non-disclosure policy has no direct effect on me, but naturally, both before and after it was promulgated, I used my own judgement to decide what should or should not be made public. The vast majority of my work I do make public (subject only to the cost of time and effort to write and explain it), because if I think something would increase risk rather than reduce it[1], then I don’t pursue this line of inquiry in the first place. Things I don’t make public are mostly early stage ideas that I don’t develop.
I think it is fair enough to judge AI alignment orgs only by the public output they produce. However, this doesn’t at all follow that a non-disclosure policy leads to immediate disqualification, like you seem to imply. You can judge an org by its public output whether or not all of its output is public. This is somewhat similar to the observation that management overhead is a bad metric. Yes, some of your money goes into something that doesn’t immediately and directly translate to benefit. All else equal, you want that not to happen. But all else is not equal, and can never be equal.
This is completely tangential, but I think we need more public discussion on how do we decide whether making something public is beneficial vs. detrimental.
Thank you for writing this impressive review!
Some comments on MIRI’s non-disclosure policy.
First, some disclosure :) My research is funded by MIRI. On the other hand, all of my opinions are my own and do not represent MIRI or anyone else associated with MIRI.
The non-disclosure policy has no direct effect on me, but naturally, both before and after it was promulgated, I used my own judgement to decide what should or should not be made public. The vast majority of my work I do make public (subject only to the cost of time and effort to write and explain it), because if I think something would increase risk rather than reduce it[1], then I don’t pursue this line of inquiry in the first place. Things I don’t make public are mostly early stage ideas that I don’t develop.
I think it is fair enough to judge AI alignment orgs only by the public output they produce. However, this doesn’t at all follow that a non-disclosure policy leads to immediate disqualification, like you seem to imply. You can judge an org by its public output whether or not all of its output is public. This is somewhat similar to the observation that management overhead is a bad metric. Yes, some of your money goes into something that doesn’t immediately and directly translate to benefit. All else equal, you want that not to happen. But all else is not equal, and can never be equal.
This is completely tangential, but I think we need more public discussion on how do we decide whether making something public is beneficial vs. detrimental.