My model is that
Alignment = an AI uses the correct model of interpretation of goals
Succeeding in this model design leads to something akin to CEV
Errors under this correct model (e.g. mesa-optimisation) are unlikely because high intelligence + a correct model of interpretation = correct extrapolation of goals
The choice of interpretation model seems almost unrelated to intelligence, as it seems equivalent to the chosen philosophies of meaning.
a) Is my model accurate?
b) Any recommendations for reading that explores alignment from a similar angle (for ex. which philosophy of meaning is most likely to emerge in LLM)? So far, Alex Flint’s posts come up the most. Which research agendas sound closest to this framing?
Thanks!
Note that your “1” has two words that both carry very heavy load—“uses” and “correct”. What does it mean for a model to be correct? How do you create one? How do you ensure that the model you implemented in software is indeed correct? How do you create AI that actually uses that model under all circumstances? In patcicular, how do you ensure that it is stable under self-improvement, out-of-distribution environments, etc? Your “2-4″ seem to indicate that you are focusing more on the “correct” part, and not enough on the “uses” part. My understanding is that if both “correct” and “uses” could be solved, it would indeed likely be a solution to the alignment problem, but it’s probably not the only path, and not necessarily the most promising one. Other paths could potentially emerge from the work on AI corrigibility, negative side-effect minimization, etc.
Yes, it’s a tough problem! :) However, your points seem to expand, rather than correct my points, which makes me think it’s not a bad way to compress the problem into a few words. Thanks!
Edit: (It seems to me that if an AI can correct for its mistakes in misinterpretation, when you look at it from the outside, it’s accurate to say it uses the correct model of interpretation, but I can see why you could disagree)