Privacy of communities isn’t a solvable problem in general, as soon as your community is large enough to compete with the adversary, it’s large enough and conspicuous enough that the adversary will pay attention to it and send in spies and extract leaks.
I disagree with this in theory as a long-term concern, but yes in practice the methods to have privacy of communities haven’t been implemented or tested at all, and I agree with the general sentiment that it isn’t worth the steep drawbacks of privacy to protect secrets, which does unfortunately make me dislike the post due to it’s strength of recommendations.
So while I could in theory disagree with you, in practice right now I mostly have to agree with the comment that there will not be such an infrastructure for private alignment ideas.
Also to touch on something here that isn’t too relevant and could be considered a tangent:
If your acceptable lower limit for basically anything is zero you wont be allowed to do anything, really anything.
This is why perfectionism is such a bad thing, and why you need to be able to accept that failure happens. You cannot have 0 failures IRL.
I have a question about this post, and it has to do with the case where both utility and error are heavy tailed:
Where does the expected value converge to if both utility and errors are heavy tailed? Is it 0, infinity, some other number, or does it not converge to any number at all?