Observations relevant to the general case where the managing algorithm can be implicit or explicit in the system:
As Eliezer (IIRC) likes to point out, correlation doesn’t happen sans shared causal origin. Correlation of utility functions is no different.
A (shared) cause doesn’t need to be temporally antecedent to its effects; it can be timeless, as seen in e.g. convergent biological evolution, aliens in unconnected Hubble volumes coming up with the same laws of physics, aliens in unconnected Hubble volumes getting the same results from their arithmetic calculators when they punch in the same numbers, etc. You can sometimes usefully spin this as teleology, as evolutionary biologists tend to do.
This doesn’t necessitate drawing a special causal arrow from the future to past. (/checks TDT document again. ) Though I’m still not seeing why you can’t use standard Bayes nets to reason about logical causes—also informally known as teleology or generally timeless conditioning. In that case calling an arrow “logical” would just be a matter of convenience. Perhaps that’s always been the case and I’d been wrongly assuming everyone was thinking something silly. Or I’m still thinking something silly ’cuz I’m missing an important point about the Markov condition or something. Anyway, point is, you can have causal arrows coming in from any direction, including the physical future. Just not the causal future, whatever that means. Determining which is which is a matter for theologians more than AI researchers.
So! Knowing that we have “parallel agents” that ended up with the same utility function, and knowing that those utility functions take a few bits to specify, we can thus infer that they have one or more common causes. For sake of simplicity we can reify those common causes into a “creator”. XDT (exceptionless decision theory) becomes relevant. We can look for scenarios/conditions where XDT is invariant under labeling of different parts of its causal history (remember, that’s not necessarily only its temporal history!) as “creator”.
I have to leave for an appointment, but anyway, that could be a place to start importing relevant intuitions from.
Observations relevant to the general case where the managing algorithm can be implicit or explicit in the system:
As Eliezer (IIRC) likes to point out, correlation doesn’t happen sans shared causal origin. Correlation of utility functions is no different.
A (shared) cause doesn’t need to be temporally antecedent to its effects; it can be timeless, as seen in e.g. convergent biological evolution, aliens in unconnected Hubble volumes coming up with the same laws of physics, aliens in unconnected Hubble volumes getting the same results from their arithmetic calculators when they punch in the same numbers, etc. You can sometimes usefully spin this as teleology, as evolutionary biologists tend to do.
This doesn’t necessitate drawing a special causal arrow from the future to past. (/checks TDT document again. ) Though I’m still not seeing why you can’t use standard Bayes nets to reason about logical causes—also informally known as teleology or generally timeless conditioning. In that case calling an arrow “logical” would just be a matter of convenience. Perhaps that’s always been the case and I’d been wrongly assuming everyone was thinking something silly. Or I’m still thinking something silly ’cuz I’m missing an important point about the Markov condition or something. Anyway, point is, you can have causal arrows coming in from any direction, including the physical future. Just not the causal future, whatever that means. Determining which is which is a matter for theologians more than AI researchers.
So! Knowing that we have “parallel agents” that ended up with the same utility function, and knowing that those utility functions take a few bits to specify, we can thus infer that they have one or more common causes. For sake of simplicity we can reify those common causes into a “creator”. XDT (exceptionless decision theory) becomes relevant. We can look for scenarios/conditions where XDT is invariant under labeling of different parts of its causal history (remember, that’s not necessarily only its temporal history!) as “creator”.
I have to leave for an appointment, but anyway, that could be a place to start importing relevant intuitions from.