It’s a question of how much does the variance in data mess up your conclusions compared to the variance in ROC.
If all the variance is in the data, then sure, several valid interpretations of the same data barely outweighs an individual with a unique data set.
However, if the data is largely shared but it’s a tough problem, so people hack at it in wildly different ways (eg outside view vs inside view), then you care more about different valid ROC than another slightly different data set.
I intended (though probably failed) to convey the idea of noninteger numbers of votes depending on the degree of correlation between datasets/ROC. If the datasets are 90% overlapping, then you dont get a full vote for adding another. If your ROC are largely overlapping (eg two attempts at outside view), then you only get a small increase in voting power, but if its large (eg inside vs outside) you can get almost another full vote.
It’s a question of how much does the variance in data mess up your conclusions compared to the variance in ROC.
If all the variance is in the data, then sure, several valid interpretations of the same data barely outweighs an individual with a unique data set.
However, if the data is largely shared but it’s a tough problem, so people hack at it in wildly different ways (eg outside view vs inside view), then you care more about different valid ROC than another slightly different data set.
I intended (though probably failed) to convey the idea of noninteger numbers of votes depending on the degree of correlation between datasets/ROC. If the datasets are 90% overlapping, then you dont get a full vote for adding another. If your ROC are largely overlapping (eg two attempts at outside view), then you only get a small increase in voting power, but if its large (eg inside vs outside) you can get almost another full vote.