This is really great for consolidating the field, thank you!
What does this mean:
You can also maintain uncertainty over the goal by trying to represent all the possible goals consistent with training data, though it’s unclear how to aggregate over the different goals.
I’m not sure how it is useful to “maintain uncertainty over the goal”?
Maintaining uncertainty over the goal allows the system to model the set of goals that are consistent with the training data, notice when they disagree with each other out of distribution, and resolve that disagreement in some way (e.g. by deferring to a human).
This is really great for consolidating the field, thank you!
What does this mean:
I’m not sure how it is useful to “maintain uncertainty over the goal”?
Thanks, glad you found the post useful!
Maintaining uncertainty over the goal allows the system to model the set of goals that are consistent with the training data, notice when they disagree with each other out of distribution, and resolve that disagreement in some way (e.g. by deferring to a human).