it seems tremendously unlikely that the brain deals with infinities in its usual mode of operation.
Unbounded is not the same as infinite. The integers are unbounded but no integer is infinite. In the same way I can have a utility function with no upper bound on the values it outputs without it ever having to output infinity.
No, it doesn’t. I think you folk are all barking up the wrong tree.
The case for unbounded utilities rests on brains actually using something like surreal numbers to represent infinite utilities. I don’t think there’s any significant evolutionary pressure favouring such a thing—or any evidence that humans actually behave that way—but at least that is a theoretical possibility.
Absent such evidence, I think Occam’s razor favours simple finite utilities that map onto the reinforcement-learning machinery evident in the brain.
Exactly. That is becaues the stuff about the finite human brain represeting unboundedly huge utilities is obvious nonsense. That is why people are roping in Omega and infinite time—desperation.
My 1,300 cm3 is capable of understanding the function f(x) = 3x, which is unbounded, therefore finite physical size does not prevent the brain from dealing with unbounded functions.
In general, a finite machine can easily deal with unbounded numbers simply by taking unbounded amounts of time to do so. This is not as much of a problem as it may sound, since there will intevitably be an upper bound to the utilities involved in all dilemma’s I actually encounter (unless my lifespan is infinite) but not the utilities I could, in theory, compute.
This is an augmented human, with a strap-on memory source bigger than the size of the planet? I thought we were probably talking about an ordinary human being—not some abstract sci-fi human that will never actually exist.
Who said anything about an augmented human, my comment was written in the first person except for one sentence, and I certainly don’t have a strap-on memory source bigger than a planet, but despite this I’m still pretty confident that I have an unbounded utility function.
Unbounded is not the same as infinite. The integers are unbounded but no integer is infinite. In the same way I can have a utility function with no upper bound on the values it outputs without it ever having to output infinity.
The human brain is limited to around 1,300 cm3. It is finite. It seems unlikely that it represents unbounded quantities for utilities.
The Peano axioms are finite. The numbers they describe are unbounded. Finite human brains understand this.
What does that have to do with how human-equivalent utility functions work?
Turing machine tapes are unbounded, but real things are not—they are finite. The human brain is finite and tiny. It is not remotely unbounded.
It shows that your conclusions from human brains being finite don’t follow.
No, it doesn’t. I think you folk are all barking up the wrong tree.
The case for unbounded utilities rests on brains actually using something like surreal numbers to represent infinite utilities. I don’t think there’s any significant evolutionary pressure favouring such a thing—or any evidence that humans actually behave that way—but at least that is a theoretical possibility.
Absent such evidence, I think Occam’s razor favours simple finite utilities that map onto the reinforcement-learning machinery evident in the brain.
Note that I never said or implied that it was.
You said:
Exactly. That is becaues the stuff about the finite human brain represeting unboundedly huge utilities is obvious nonsense. That is why people are roping in Omega and infinite time—desperation.
My 1,300 cm3 is capable of understanding the function f(x) = 3x, which is unbounded, therefore finite physical size does not prevent the brain from dealing with unbounded functions.
In general, a finite machine can easily deal with unbounded numbers simply by taking unbounded amounts of time to do so. This is not as much of a problem as it may sound, since there will intevitably be an upper bound to the utilities involved in all dilemma’s I actually encounter (unless my lifespan is infinite) but not the utilities I could, in theory, compute.
This is an augmented human, with a strap-on memory source bigger than the size of the planet? I thought we were probably talking about an ordinary human being—not some abstract sci-fi human that will never actually exist.
Who said anything about an augmented human, my comment was written in the first person except for one sentence, and I certainly don’t have a strap-on memory source bigger than a planet, but despite this I’m still pretty confident that I have an unbounded utility function.