Thanks for the links! I was unaware of these and both are interesting.
I was probably a little heavy-handed in my wording in the post, I agree with Shalizi’s comment that we should be careful to over-interpret the analogy between physics and Bayesian analysis. However, my goal isn’t to “derive physics from Bayesian analysis” it’s more of a source of inspiration. Physics tells us that continuous symmetries lead to robust conservation laws, so because the mathematics is so similar, if we could force the reward functions to exhibit the same invariance (Noether currents, conservation laws etc..) then , by analogy with the robust generalisation in physics—it might help AI generalise reliably even out of distribution.
If I understand Nguyen’s critique there are essentially two parts:
a) The gauge theory reformulation is mathematically valid, but trivial, and can be reformulated without gauge theory. Furthermore, he claims that because Weinstein is using such a complex mathematical formalism for doing something trivial he risks obscuritanism.
My response: I think it’s unlikely that current reward functions are trivially invariant under gauge transformations of the moral coordinates. There are many examples where we have respectable moral frameworks with genuine moral disagreement on certain statements. Current approaches seek to “average over” these disagreements rather than translate between them in an invariant way.
b) The theory depends on choice of a connection (in my formulation Aμ) which is not canonical. In other words, it’s not clear what choice would capture “true” moral behaviour.
My response: I agree that this is challenging (which is part of the reason I didn’t try to do it in the post.) However, I think the difficulty is valuable. If our reward function cannot be endowed with these canonical invariances it won’t generalise robustly out of distribution. In that sense, using these ideas as a kind of diagnostic tool to assess whether the reward function possesses some invariance could give us a clue about whether the reward will generalise robustly.
There has been a minor advance in professional engagement with Malaney and Weinstein’s gauge theory of economics; the economist Robert Murphy had Weinstein on his podcast, and wrote a few blog posts about it. There’s nothing formal here, but the informal discussion is essential if one is to understand how gauge theory could be relevant in an economic context, and the same is true if it is going to be successfully imported into decision theory and cognitive science.
Thanks for the links! I was unaware of these and both are interesting.
I was probably a little heavy-handed in my wording in the post, I agree with Shalizi’s comment that we should be careful to over-interpret the analogy between physics and Bayesian analysis. However, my goal isn’t to “derive physics from Bayesian analysis” it’s more of a source of inspiration. Physics tells us that continuous symmetries lead to robust conservation laws, so because the mathematics is so similar, if we could force the reward functions to exhibit the same invariance (Noether currents, conservation laws etc..) then , by analogy with the robust generalisation in physics—it might help AI generalise reliably even out of distribution.
If I understand Nguyen’s critique there are essentially two parts:
a) The gauge theory reformulation is mathematically valid, but trivial, and can be reformulated without gauge theory. Furthermore, he claims that because Weinstein is using such a complex mathematical formalism for doing something trivial he risks obscuritanism.
My response: I think it’s unlikely that current reward functions are trivially invariant under gauge transformations of the moral coordinates. There are many examples where we have respectable moral frameworks with genuine moral disagreement on certain statements. Current approaches seek to “average over” these disagreements rather than translate between them in an invariant way.
b) The theory depends on choice of a connection (in my formulation Aμ) which is not canonical. In other words, it’s not clear what choice would capture “true” moral behaviour.
My response: I agree that this is challenging (which is part of the reason I didn’t try to do it in the post.) However, I think the difficulty is valuable. If our reward function cannot be endowed with these canonical invariances it won’t generalise robustly out of distribution. In that sense, using these ideas as a kind of diagnostic tool to assess whether the reward function possesses some invariance could give us a clue about whether the reward will generalise robustly.
There has been a minor advance in professional engagement with Malaney and Weinstein’s gauge theory of economics; the economist Robert Murphy had Weinstein on his podcast, and wrote a few blog posts about it. There’s nothing formal here, but the informal discussion is essential if one is to understand how gauge theory could be relevant in an economic context, and the same is true if it is going to be successfully imported into decision theory and cognitive science.