I’m interested in this line of reasoning. I can’t really say much in response right now, but I just read that paper you linked—they write such clear and easily, heh, interpretable papers don’t they? - and I have strong opinions about “the correct value system” being rooted in maximizing some weighted sum of the “autonomy” of all living / agentic / intelligent systems, which it seems like you’re gesturing towards as well. I’m interested in trying to figure out how to formalize this.
I’m interested in this line of reasoning. I can’t really say much in response right now, but I just read that paper you linked—they write such clear and easily, heh, interpretable papers don’t they? - and I have strong opinions about “the correct value system” being rooted in maximizing some weighted sum of the “autonomy” of all living / agentic / intelligent systems, which it seems like you’re gesturing towards as well. I’m interested in trying to figure out how to formalize this.