AI Benchmarking

TagLast edit: 16 Jul 2023 14:12 UTC by rybolos

Broken Benchmark: MMLU

awg29 Aug 2023 18:09 UTC

24 points

5 comments1 min readLW link

(www.youtube.com)

LLM Psychometrics: A Speculative Approach to AI Safety

pskl29 Jan 2024 18:38 UTC

3 points

4 comments1 min readLW link

(pascal.cc)

Auto-Enhance: Developing a meta-benchmark to measure LLM agents’ ability to improve other agents

Sam F. Brown, BasilLabib, Codruta (Coco) Lugoj and Sai Sasank Y

22 Jul 2024 12:33 UTC

20 points

0 comments14 min readLW link

Improving Model-Written Evals for AI Safety Benchmarking

Sunishchal Dev and Marius Hobbhahn

15 Oct 2024 18:25 UTC

27 points

0 comments18 min readLW link

Workshop Report: Why current benchmarks approaches are not sufficient for safety?

Tom DAVID and Pierre Peigné

26 Nov 2024 17:20 UTC

3 points

1 comment3 min readLW link

Introducing REBUS: A Robust Evaluation Benchmark of Understanding Symbols

Arjun Panickssery and agg

15 Jan 2024 21:21 UTC

33 points

0 comments1 min readLW link

MMLU’s Moral Scenarios Benchmark Doesn’t Measure What You Think it Measures

corey morris27 Sep 2023 17:54 UTC

18 points

2 comments4 min readLW link

(medium.com)

No comments.