Gurkenglas comments on Reproducing ARC Evals’ recent report on language model agents

[Gurkenglas]

If it is twice as easy, that halves the positives of open-sourcing and the negatives, it doesn’t change the direction.

Beware the Unilateralist’s Curse.