What is the difference between Evaluation, Characterization, Experiments, and Observations?
The words evaluations, experiments, characterizations, and observations are somewhat confused or confusingly used in discussions about model evaluations (e.g., ref, ref).
Let’s define them more clearly:
Observations provide information about an object (including systems).
This information can be informative (allowing the observer to update its beliefs significantly), or not.
Characterizations describe distinctive features of an object (including properties).
Characterizations are observations that are actively designed and controlled to study an object.
Evaluations evaluate the quality of distinctive features based on normative criteria.
Evaluations are composed of both characterizations and normative criteria.
Evaluations are normative, they inform about what is good or bad, desirable or undesirable.
Normative criteria (or “evaluation criterion”) are the element bringing the normativity. They are most of the time directional or simple thresholds.
Evaluations include both characterizations of the object studied and characterization of the characterization technique used (e.g., accuracy of measurement).
Scientific experiments test hypotheses through controlled manipulation of variables.
Scientific experiments are composed of: characterizations, and hypothesis
In summary:
Observations
Characterizations = Designed and controlled Observations
Evaluations = Characterization of object + Characterization of the characterization method + Normative criteria
An observation is an event in which the observer receives information about the AI system.
E.g., you read a completion returned by a model.
A characterization is a tool or process used to describe an AI system.
E.g., you can characterize the latency of an AI system by measuring it. You can characterize how often a model is correct (without specifying that correctness is the goal).
An AI system evaluation will associate characterizations and normative criteria to conclude about the quality of the AI system on the dimensions evaluated.
E.g., alignment evaluations use characterizations of models and the normative criteria of the alignment with X (e.g., humanity) to conclude on how well the model is aligned with X.
An experiment will associate hypotheses, interventions, and finally characterizations to conclude on the veracity of the hypotheses about the AI system.
E.g., you can change the training algorithm and measure the impact using characterization techniques.
Clash of usage and definition:
These definitions slightly clash with the usage of the term evals or evaluations in the AI community. Regularly the normative criteria associated with an evaluation are not explicitly defined, and the focus is solely put on the characterizations included in the evaluation.
(Produced as part of the AI Safety Camp, within the project: Evaluating alignment evaluations)
What is the difference between Evaluation, Characterization, Experiments, and Observations?
The words evaluations, experiments, characterizations, and observations are somewhat confused or confusingly used in discussions about model evaluations (e.g., ref, ref).
Let’s define them more clearly:
Observations provide information about an object (including systems).
This information can be informative (allowing the observer to update its beliefs significantly), or not.
Characterizations describe distinctive features of an object (including properties).
Characterizations are observations that are actively designed and controlled to study an object.
Evaluations evaluate the quality of distinctive features based on normative criteria.
Evaluations are composed of both characterizations and normative criteria.
Evaluations are normative, they inform about what is good or bad, desirable or undesirable.
Normative criteria (or “evaluation criterion”) are the element bringing the normativity. They are most of the time directional or simple thresholds.
Evaluations include both characterizations of the object studied and characterization of the characterization technique used (e.g., accuracy of measurement).
Scientific experiments test hypotheses through controlled manipulation of variables.
Scientific experiments are composed of: characterizations, and hypothesis
In summary:
Observations
Characterizations = Designed and controlled Observations
Evaluations = Characterization of object + Characterization of the characterization method + Normative criteria
Scientific experiments = Characterizations + Hypothesis
Examples:
An observation is an event in which the observer receives information about the AI system.
E.g., you read a completion returned by a model.
A characterization is a tool or process used to describe an AI system.
E.g., you can characterize the latency of an AI system by measuring it. You can characterize how often a model is correct (without specifying that correctness is the goal).
An AI system evaluation will associate characterizations and normative criteria to conclude about the quality of the AI system on the dimensions evaluated.
E.g., alignment evaluations use characterizations of models and the normative criteria of the alignment with X (e.g., humanity) to conclude on how well the model is aligned with X.
An experiment will associate hypotheses, interventions, and finally characterizations to conclude on the veracity of the hypotheses about the AI system.
E.g., you can change the training algorithm and measure the impact using characterization techniques.
Clash of usage and definition:
These definitions slightly clash with the usage of the term evals or evaluations in the AI community. Regularly the normative criteria associated with an evaluation are not explicitly defined, and the focus is solely put on the characterizations included in the evaluation.
(Produced as part of the AI Safety Camp, within the project: Evaluating alignment evaluations)