No matter what kind of program you create, no matter how cleverly you spin it or complexify or simplify or reduce it, there will always, by logical necessity, be some subset of it that you can point at and say “Look here! This is what ‘determines’ what experimental results I see and restricts the possible futures! Let’s call this thinghy/subset/formula ‘reality’!”
I don’t see any possibility of getting around that requirement unless I assume magic, supernatural entities, wishful thinking, ontologically basic nonlogical entities, or worse.
As far as I can tell, those two paragraphs are pretty much Eliezer’s position on this, and he’s just putting that subset as an arbitrary variable, saying something like “Sure, we might not know said subset of the program or where exactly it is or what computational form it takes, but let’s just have a name for it anyway so we can talk about things more easily”.
As far as I can tell, those two paragraphs are pretty much Eliezer’s position on this, and he’s just putting that subset as an arbitrary variable, saying something like “Sure, we might not know said subset of the program or where exactly it is or what computational form it takes, but let’s just have a name for it anyway so we can talk about things more easily”.