We’re open-sourcing OpenAI Evals, our software framework for creating and running benchmarks for evaluating models like GPT-4, while inspecting their performance sample by sample. We invite everyone to use Evals to test our models and submit the most interesting examples.
Someone should submit the few safety benchmarks we have if they haven’t been submitted already, including things like:
Someone should submit the few safety benchmarks we have if they haven’t been submitted already, including things like:
Model-Written Evals (especially the “advanced AI risk” evaluations)
Helpful, Honest, & Harmless Alignment
ETHICS
Am I missing others that are straightforward to submit?