Has anyone thought about the things that governments are uniquely good at when it comes to evaluating models?
Here are at least 3 things I think they have as benefits: 1. Just an independent 3rd-party perspective generally
2. The ability to draw insights across multiple labs’ efforts, and identify patterns that others might not be able to
3. The ability to draw on classified threat intelligence to inform its research (e.g., Country X is using model Y for bad behavior Z) and to test the model for classified capabilities (bright line example: “can you design an accurate classified nuclear explosive lensing arrangement”).
Has anyone thought about the things that governments are uniquely good at when it comes to evaluating models?
Here are at least 3 things I think they have as benefits:
1. Just an independent 3rd-party perspective generally
2. The ability to draw insights across multiple labs’ efforts, and identify patterns that others might not be able to
3. The ability to draw on classified threat intelligence to inform its research (e.g., Country X is using model Y for bad behavior Z) and to test the model for classified capabilities (bright line example: “can you design an accurate classified nuclear explosive lensing arrangement”).
Are there others that come to mind?