This illustrates how sample efficiency is another way of framing robust off-distribution behavior, the core problem of alignment. Higher sample efficiency means that on-distribution you need to fill fewer gaps between the data points, that data points can be more spaced out while still conveying their intent. This suggests that at the border of the training distribution, you can get further away without breaking robustness if there’s higher sample efficiency.
And the ultimate problem of alignment is unbounded reflection/extrapolation off-distribution, which corresponds to unbounded sample efficiency. I mean this in the somewhat noncentral sense where AlphaZero has unbounded sample efficiency, as it doesn’t need to look at any external examples to get good at playing. But it also doesn’t need to remain aligned with anything, so not a complete analogy. IDA seeks to make the analogy stronger.
This illustrates how sample efficiency is another way of framing robust off-distribution behavior, the core problem of alignment. Higher sample efficiency means that on-distribution you need to fill fewer gaps between the data points, that data points can be more spaced out while still conveying their intent. This suggests that at the border of the training distribution, you can get further away without breaking robustness if there’s higher sample efficiency.
And the ultimate problem of alignment is unbounded reflection/extrapolation off-distribution, which corresponds to unbounded sample efficiency. I mean this in the somewhat noncentral sense where AlphaZero has unbounded sample efficiency, as it doesn’t need to look at any external examples to get good at playing. But it also doesn’t need to remain aligned with anything, so not a complete analogy. IDA seeks to make the analogy stronger.