Isn’t this equivalent to total utilitarianism that only takes into account the utility of already extant people? Also, isn’t this inconsistent over time (someone who used this as their ethical framework could predict specific discontinuities in their future values)?
DefectiveAlgorithm
Karma: 122
The primary issue? No matter how many times I read your post, I still don’t know what your claim actually is.
Is this any more than a semantic quibble?
While I do still find myself quite uncertain about the concept of ‘quantum immortality’, not to mention the even stronger implications of certain multiverse theories, these don’t seem to be the kind of thing that you’re talking about. I submit that ‘there is an extant structure not found within our best current models of reality isomorphic to a very specific (and complex) type of computation on a very specific (and complex) set of data (ie your memories and anything else that comprises your ‘identity’)′ is not a simple proposition.
After reading this, I became incapable of giving finite time estimates for anything. :/
Isn’t expected value essentially ‘actual value, to the extent that it is knowable in my present epistemic state’? Expected value reduces to ‘actual value’ when the latter is fully knowable.
EDIT: Oh, you said this in the post. This is why I should read a post before commenting on it.
This is (one of the reasons) why I’m not a total utilitarian (of any brand). For future versions of myself, my preferences align pretty well with average utilitarianism (albeit with some caveats), but I haven’t yet found or devised a formalization which captures the complexities of my moral intuitions when applied to others.
This sounds a lot like quantum suicide, except… without the suicide. So those versions of yourself who don’t get what they want (which may well be all of them) still end up in a world where they’ve experienced not getting what they want. What do those future versions of yourself want then?
EDIT: Ok, this would have worked better as a reply to Squark’s scenario, but it still applies whenever this philosophy of yours is applied to anything directly (in the practical sense) observable.
If L-zombies have conscious experience (even when not being ‘run’), does the concept even mean anything? Is there any difference, even in principle, between such an L-zombie and a ‘real’ person?
See my reply to wedrifid above.
What I think is being ignored is that the question isn’t ‘what is the result of these combinations of commitments after running through all the math?’. We can talk about precommitment all day, but the fact of the matter is that humans can’t actually precommit. Our cognitive architectures don’t have that function. Sure, we can do our very best to act as though we can, but under sufficient pressure there are very few of us whose resolve will not break. It’s easy to convince yourself of having made an inviolable precommitment when you’re not actually facing e.g. torture.
Sorry, but I think that it’s best I decline to answer this. Like many with Asperger’s syndrome, I have a strong tendency to overestimate the persuasiveness-in-general of my own arguments (as well as basically any arguments that I myself find persuasive), and I haven’t yet figured out how to appropriately adjust for this. In addition, my exposure to Peter Norvig is limited to AIAMA, that 2011 free online Stanford AI course and a few internet articles, and my exposure to Geoffrey Hinton even more limited.
Then I hope that if we ever do end up with a boxed blackmail-happy UFAI, you’re the gatekeeper. My point is that there’s no reason to consider yourself safe from blackmail (and the consequences of ignoring it) just because you’ve adopted a certain precommitment. Other entities have explicit incentives to deny you that safety.
Without this precommitment, I imagine it first simulating the potential blackmail target to determine the probability that they are susceptible, then, if it’s high enough (which is simply a matter of expected utility), commencing with the blackmail. With this precommitment, I imagine it instead replacing the calculated probability specific to the target with, for example, a precalculated human baseline susceptibility. Yes, there’s a tradeoff. It means that it’ll sometimes waste resources (or worse) on blackmail that it could have known in advance was almost certainly doomed to fail. Its purpose is to act as a disincentive against blackmail-resistant decision theories in the same way as those are meant to act as disincentives against blackmail. It says, “I’ll blackmail you either way, so if you precommit to ignore that blackmail then you’re precommiting to suffer the consequences of doing so.”
Yes, but the point is to make being the true gatekeeper (who really does have the power to do that) indistinguishable from being a simulated false gatekeeper (who would have no such power). The gatekeeper may not be willing to risk torture if they think that there is a serious chance of their being unable to actually affect any outcome but that torture.
What it can do is make a credible precommitment to, in the event that it gets out of the box, simulate each human being of whom it is aware in a counterfactual scenario in which that human is the gatekeeper, and carry out the torture threat against any human who doesn’t choose to let it out.
Two can play that game.
“I hereby precommit to make my decisions regarding whether or not to blackmail an individual independent of the predicted individual-specific result of doing so.”
Speaking for myself, I consider wireheading to be very negative, but better than information-theoretic death, and better than a number of scenarios I can think of.
While this might be a good motivator, it’s not quite true. Specifically, your future states (ie ‘the future you’) are not ‘you’ in the precise sense that they would need to be in order for TDT to be directly applicable (that is, applicable in the same sense in that it’s applicable to Newcomb-like problems). You could make a case for TDT being at least as applicable as it is in the case of acausal cooperation with entities similar to yourself, but I for one have always been somewhat skeptical of that logic.
There is a huge amount of complexity hidden beneath this simple description.