When they talk about an expert benchmark, they’re talking about the performance of a human expert right? That’s what it sounds like, but it’s worth being sure.
No, I’m pretty confident every expert is a neural network policy trained on the task. See “F. Data Collection Details” and the second paragraph of “3.3. Robotics—RGB Stacking Benchmark (real and sim)”
When they talk about an expert benchmark, they’re talking about the performance of a human expert right? That’s what it sounds like, but it’s worth being sure.
No, I’m pretty confident every expert is a neural network policy trained on the task. See “F. Data Collection Details” and the second paragraph of “3.3. Robotics—RGB Stacking Benchmark (real and sim)”
I read the paper and this is correct.
That is how I have seen the term expert performance used in papers. Yes.
What would then 50% of expert performance mean? If 100% is a human expert and 0% is a random policy then what is the meaning of something in between?
Getting half the score, getting half as many questions right, etc.