I would further add that looking for difficulties created by the simplification seems very intellectually productive. (Solving “embedded agency problems” seems to genuinely allow you to do new things, rather than just soothing philosophical worries.) But yeah, I would agree that if we’re defining mesa-objective anyway, we’re already in the business of assuming some agent/environment boundary.
I would further add that looking for difficulties created by the simplification seems very intellectually productive.
Yep, strongly agree. And a good first step to doing this is to actually build as robust a simplification as you can, and then see where it breaks. (Working on it.)
I would further add that looking for difficulties created by the simplification seems very intellectually productive. (Solving “embedded agency problems” seems to genuinely allow you to do new things, rather than just soothing philosophical worries.) But yeah, I would agree that if we’re defining mesa-objective anyway, we’re already in the business of assuming some agent/environment boundary.
Yep, strongly agree. And a good first step to doing this is to actually build as robust a simplification as you can, and then see where it breaks. (Working on it.)