Also, we typically think of utility-maximizers as wanting to decrease entropy. There’s various attempts at formalizing this, e.g. Utility Maximization = Description Length Minimization. Could this account for some of the difficulty in alignment? Like if life wants to maximize entropy, and utility maximizers want to minimize entropy, that seems like a natural conflict. But there’s a bunch of technical details that makes me unsure whether this logical actually works out.
Also, we typically think of utility-maximizers as wanting to decrease entropy. There’s various attempts at formalizing this, e.g. Utility Maximization = Description Length Minimization. Could this account for some of the difficulty in alignment? Like if life wants to maximize entropy, and utility maximizers want to minimize entropy, that seems like a natural conflict. But there’s a bunch of technical details that makes me unsure whether this logical actually works out.