I didn’t remember that test from earlier, either. Worth checking out? I don’t mind accidentally unblinding a little if it is an experimental/control difference—curious folks will be curious.
I just went through the whole thing again; there was no test of that kind at the end. (What there was was the previous multiple-choice quiz about some example forecasts and how they went wrong.) Looks like this is an experimental/control difference. I’d rather not discuss that bit further—this isn’t about possibly life-or-death drugs, after all, and I already know where I can find calibration tests like that.
I’ve been wondering if it’s not the other way round, the Good Judgement project copying Inkling Market’s questions? What info do you have that leads you to think the copying was in the direction you assume?
My evidence for the other way round is that the Brent question has a starred footnote which is present on IM but not on GJ, while the star is in the text of the GJ question.
First, for those of you who logged in before September 6, please be aware that the tournament’s sponsor issued 10 new questions today, which we have posted. As the About page notes, new questions usually will be distributed on Mondays (but not every Monday). These questions arrived on Tuesday because of the Labor Day holiday.
My understanding was that the sponsor was IARPA. And googling, I don’t see any listed connections between Inkling and Good Judgement Project.
Stray asterisks are very suspicious. I see one in the Inkling question, but I don’t see the footnote itself. It has a “background information” section, but all their questions do. Is “last price” a technical term? If the usual term is “settlement price,” and GJ doesn’t make that clear, then it is quite suspicious.
Here aretwo two more Inkling questions with asterisks. One has an explicit footnote. The other is a change to the question.
I didn’t remember that test from earlier, either. Worth checking out? I don’t mind accidentally unblinding a little if it is an experimental/control difference—curious folks will be curious.
I just went through the whole thing again; there was no test of that kind at the end. (What there was was the previous multiple-choice quiz about some example forecasts and how they went wrong.) Looks like this is an experimental/control difference. I’d rather not discuss that bit further—this isn’t about possibly life-or-death drugs, after all, and I already know where I can find calibration tests like that.
Fine with me. :)
BTW, look what I found. Did you know about this one?
Looks like someone is being very naughty. I’ve asked him on Twitter.
I’ve been wondering if it’s not the other way round, the Good Judgement project copying Inkling Market’s questions? What info do you have that leads you to think the copying was in the direction you assume?
My evidence for the other way round is that the Brent question has a starred footnote which is present on IM but not on GJ, while the star is in the text of the GJ question.
The description, from the latest email, is
My understanding was that the sponsor was IARPA. And googling, I don’t see any listed connections between Inkling and Good Judgement Project.
Stray asterisks are very suspicious. I see one in the Inkling question, but I don’t see the footnote itself. It has a “background information” section, but all their questions do. Is “last price” a technical term? If the usual term is “settlement price,” and GJ doesn’t make that clear, then it is quite suspicious.
Here are two two more Inkling questions with asterisks. One has an explicit footnote. The other is a change to the question.