I love the framing of outer alignment as a data quality problem!
As an illustrative data point, the way Google generates “alignment data” for its search evals is by employing thousands of professional raters and training them to follow a 200-page handbook (!) that operationalizes the concept of a “good search result”.
I love the framing of outer alignment as a data quality problem!
As an illustrative data point, the way Google generates “alignment data” for its search evals is by employing thousands of professional raters and training them to follow a 200-page handbook (!) that operationalizes the concept of a “good search result”.