Some people (including me) have made comments along these lines before. There’s nothing theoretically wrong with the view that evolutionary history may have created multiple less-than-coordnated utility functions that happen to share one brain.
The consequences have some serious implications, though. If a single human has multiple utility functions, it is highly unlikely (for reasons similar to Arrow’s Paradox) that these work out compatibly enough that you can have an organism-wide utility expressed as a real number (as opposed to a hypercomplex number or matrix). And if you have to map utility to a hypercomplex number or matrix, you can’t “shut up and multiply”, because while 7*3^^^3 is always a really big number, matrix math is a lot more complicated. Utilitarianism becomes mathematically intractable as a result.
Some people (including me) have made comments along these lines before. There’s nothing theoretically wrong with the view that evolutionary history may have created multiple less-than-coordnated utility functions that happen to share one brain.
The consequences have some serious implications, though. If a single human has multiple utility functions, it is highly unlikely (for reasons similar to Arrow’s Paradox) that these work out compatibly enough that you can have an organism-wide utility expressed as a real number (as opposed to a hypercomplex number or matrix). And if you have to map utility to a hypercomplex number or matrix, you can’t “shut up and multiply”, because while 7*3^^^3 is always a really big number, matrix math is a lot more complicated. Utilitarianism becomes mathematically intractable as a result.