Do you mean that as in “you can describe/encode arbitrary systems as a single number” or something related to that?
Yes.
For my part, I also consider it perfectly plausible (though perhaps less likely than some alternatives) that some humans might actually have tiered systems where certain values really truly never can be traded off in the slightest fraction of opportunity costs against arbitrarily high values of all lower-tiered values at the same time.
For instance, I could imagine an agent that values everything I value but has a hard tier cutoff below the single value that its consciousness must remain continuously aware until the end of the universe if such a time ever arrives (forever otherwise, assuming the simplest alternative). This agent would have no trouble sacrificing the entire solar system if it was proven to raise the expected odds of this survival. Or the agent could also only have to satisfy a soft threshold or some balancing formula where a certain probability of eternal life is desired, but more certainty than that becomes utility-comparable to lower-tier values. Or many other kinds of possible constructs.
So yes, arbitrary systems, for all systems I’ve ever thought of. I like to think of myself as imaginative and as having thought of a lot of possible arbitrary systems, too, though obviously my search space is limited by my intelligence and by the complexity I can formulate.
There are actual tiered systems all around us, even if most examples that come to mind are constructed/thought of by humans.
That aside, I am claiming that I would not trade my highest tier values against arbitrary combinations of all lower tiered values. So … hi!
Re: Just a number; I can encode your previous comments (all of them) in the form of a bitstring, which is a number. Doesn’t mean that doing “+1” on that yields any sensible result. Maybe we’re talking past each other on the “describe/encode” point, but I don’t see how describing a system containing strict tiers as a number somehow makes those tiers go away, unless you were nitpicking about “everything’s just a number that’s interpreted in a certain way” or somesuch.
Ah, on the numbers thing, what I meant was only that AFAIK there always exists some formula for which higher output numbers will correspond to things any abitrary agent (at least, all the logically valid and sound ones that I’ve thought of) would prefer.
So even for a hard tier system, there’s a way to compute a number linearly representative of how happy the agent is with worldstates, where at the extreme all lower-tier values flatline into arbitrarily large negatives (or other, more creative / leakproof weighing) whenever they incur infinitesimal risk of opportunity cost towards the higher-tier values.
The reason I’m said this is because it’s often disputed and/or my audience isn’t aware of it, and I often have to prove even the most basic versions of this claim (such as “you can represent a tiered system where as soon as the higher tier is empty, the lower tier is worthless using a relatively simple mathematical formula”) by showing them the actual equations and explaining how it works.
I don’t understand. Do you mean that as in “you can describe/encode arbitrary systems as a single number” or something related to that?
If not, do you mean that there must be some number of sparrows outweighing everything else as it gets sufficiently large?
Please explain.
Yes.
For my part, I also consider it perfectly plausible (though perhaps less likely than some alternatives) that some humans might actually have tiered systems where certain values really truly never can be traded off in the slightest fraction of opportunity costs against arbitrarily high values of all lower-tiered values at the same time.
For instance, I could imagine an agent that values everything I value but has a hard tier cutoff below the single value that its consciousness must remain continuously aware until the end of the universe if such a time ever arrives (forever otherwise, assuming the simplest alternative). This agent would have no trouble sacrificing the entire solar system if it was proven to raise the expected odds of this survival. Or the agent could also only have to satisfy a soft threshold or some balancing formula where a certain probability of eternal life is desired, but more certainty than that becomes utility-comparable to lower-tier values. Or many other kinds of possible constructs.
So yes, arbitrary systems, for all systems I’ve ever thought of. I like to think of myself as imaginative and as having thought of a lot of possible arbitrary systems, too, though obviously my search space is limited by my intelligence and by the complexity I can formulate.
There are actual tiered systems all around us, even if most examples that come to mind are constructed/thought of by humans.
That aside, I am claiming that I would not trade my highest tier values against arbitrary combinations of all lower tiered values. So … hi!
Re: Just a number; I can encode your previous comments (all of them) in the form of a bitstring, which is a number. Doesn’t mean that doing “+1” on that yields any sensible result. Maybe we’re talking past each other on the “describe/encode” point, but I don’t see how describing a system containing strict tiers as a number somehow makes those tiers go away, unless you were nitpicking about “everything’s just a number that’s interpreted in a certain way” or somesuch.
Ah, on the numbers thing, what I meant was only that AFAIK there always exists some formula for which higher output numbers will correspond to things any abitrary agent (at least, all the logically valid and sound ones that I’ve thought of) would prefer.
So even for a hard tier system, there’s a way to compute a number linearly representative of how happy the agent is with worldstates, where at the extreme all lower-tier values flatline into arbitrarily large negatives (or other, more creative / leakproof weighing) whenever they incur infinitesimal risk of opportunity cost towards the higher-tier values.
The reason I’m said this is because it’s often disputed and/or my audience isn’t aware of it, and I often have to prove even the most basic versions of this claim (such as “you can represent a tiered system where as soon as the higher tier is empty, the lower tier is worthless using a relatively simple mathematical formula”) by showing them the actual equations and explaining how it works.