I still don’t think you could judge a thousand such trials and be wrong only once.
When I first saw this, I agreed with it. But now I don’t, partly because of the story (which I don’t have a link to, but it was linked to from LW somewhere) about someone who would bet they knew whether or not a number was a prime. This continued until they made a mistake (doing it mentally), and then they lost.
If they had a calculator, could they go up to the 1000th odd number and be wrong at most once? I’m pretty sure they could, actually. And so the question isn’t “can you judge 1000 trials and only get one wrong?” but “can you judge 1000 obvious trials and only get one wrong?”, or, more appropriately, “can you judge 1000 trials as either ‘obvious’ and ‘contested’ and only be wrong at most once?”. Because originally I was imagining being a normal trial judge- but a normal trial judge has to deal with difficult cases. Ones like the Amanda Knox case (are/should be) rare. I’m pretty confident that once you put in a reasonable amount of effort (however much komponisto did for this case), you can tell whether or not the case is one you can be confident about or one you can’t, assuming you’re carefully thinking about what would make them not open-and-shut cases.
When I first saw this, I agreed with it. But now I don’t, partly because of the story (which I don’t have a link to, but it was linked to from LW somewhere) about someone who would bet they knew whether or not a number was a prime. This continued until they made a mistake (doing it mentally), and then they lost.
If they had a calculator, could they go up to the 1000th odd number and be wrong at most once? I’m pretty sure they could, actually. And so the question isn’t “can you judge 1000 trials and only get one wrong?” but “can you judge 1000 obvious trials and only get one wrong?”, or, more appropriately, “can you judge 1000 trials as either ‘obvious’ and ‘contested’ and only be wrong at most once?”. Because originally I was imagining being a normal trial judge- but a normal trial judge has to deal with difficult cases. Ones like the Amanda Knox case (are/should be) rare. I’m pretty confident that once you put in a reasonable amount of effort (however much komponisto did for this case), you can tell whether or not the case is one you can be confident about or one you can’t, assuming you’re carefully thinking about what would make them not open-and-shut cases.