A lot of this is going to come down to estimates of the denominator.
(I mostly just think that you might as well just ask people “Is this good?” rather than trying to use a more sophisticated form of IRL—in particular I don’t think that realistic versions of IRL will successfully address the cases where people err in answering the “is it good?” question, that directly asking is more straightforward in many important ways, and that we should mostly just try to directly empower people to give better answers to such questions.)
Anyway, with that caveat and kind of using the version of your idea that I feel most enthusiastic about (and construing it quite broadly), I have a significant probability on 0, maybe a median somewhere in 10-20, significant probability at very high levels.
A lot of this is going to come down to estimates of the denominator.
(I mostly just think that you might as well just ask people “Is this good?” rather than trying to use a more sophisticated form of IRL—in particular I don’t think that realistic versions of IRL will successfully address the cases where people err in answering the “is it good?” question, that directly asking is more straightforward in many important ways, and that we should mostly just try to directly empower people to give better answers to such questions.)
Anyway, with that caveat and kind of using the version of your idea that I feel most enthusiastic about (and construing it quite broadly), I have a significant probability on 0, maybe a median somewhere in 10-20, significant probability at very high levels.