An high-Kolmogorov-complexity system is still a system.
Paperclip Minimizer
Karma: 122
I’m not sure what it would even mean to not have a Real Moral System. The actual moral judgments must come from somewhere.
Using PCA on utility functions could be an interesting research subject for wannabe AI risk experts.
I don’t see the argument. I have an actual moral judgement that painless extermination of all sentient beings is evil, and so is tiling the universe with meaningless sentient beings.
don’t trust studies that would be covered in the Weird News column of the newspaper
-- Ozy
Good post. Some nitpicks:
There are many models of rationality from which a hypothetical human can diverge, such as VNM rationality of decision making, Bayesian updating of beliefs, certain decision theories or utilitarian branches of ethics. The fact that many of them exist should already be a red flag on any individual model’s claim to “one true theory of rationality.”
VNM rationality, Bayesian updating, decision theories, and utilitarian branches of ethics all cover different areas. They aren’t incompatible and actually fit rather neatly into each other.
As a Jacobin piece has pointed out
This is a Jacobite piece.
A critique of Pro Publica is not meant to be an endorsement of Bayesian justice system, which is still a bad idea due to failing to punish bad actions instead of things correlated with bad actions.
Unless you’re omniscient, you can only punish things correlated with bad actions.
While this may seem like merely a niche issue, given the butterfly effect and a sufficiently long timeline with the possibility of simulations, it is almost guaranteed that any decision will change.
I think you accidentally words.
.
Noticing an unachievable goal may force it to have an existential crisis of sorts, resulting in self-termination.
Do you have reasoning behind this being true, or is this baseless anthropomorphism ?
It should not hurt an aligned AI, as it by definition conforms to the humans’ values, so if it finds itself well-boxed, it would not try to fight it.
So it is an useless AI ?
Your whole comment is founded on a false assumption. Look at Bayes’ formula. Do you see any mention of whether your probability estimate is “just your prior” or “the result of a huge amount of investigation and very strong reasoning” ? No ? Well this mean that this doesn’t effect how much you’ll update.
“self-aware” can also be “self-aware” as in, say, “self-aware humor”
I don’t see why negative utilitarians would be more likely than positive utilitarians to support animal-focused effective altruism over (near-term) human-focused effective altruism.
This actually made me not read the whole sequence.
[1] It would be rather audacious to claim that this is true for each of the four axioms. For instance, do please demonstrate how you would Dutch-book an agent that does not conform to the completeness axiom!
How can an agent not conform the completeness axiom ? It literally just say “either the agent prefer A to B, or B to A, or don’t prefer anything”. Offer me an example of an agent that don’t conform to the completeness axiom.
Obviously it’s true that we face trade-offs. What is not so obvious is literally the entire rest of the section I quoted.
The entire rest of the section is a straightforward application of the theorem. The objection is that X don’t happen in real life, and the counter-objection is that something like X do happen in real life, meaning the theorem do apply.
As I explained above, the VNM theorem is orthogonal to Dutch book theorems, so this response is a non sequitur.
Yeah, sorry for being imprecise in my language. Can you just be charitable and see that my statement make sense if you replace “VNM” by “Dutch book” ? Your behavior does not really send the vibe of someone who want to approach this complicated issue honestly, and more send the vibe of someone looking for Internet debate points.
More generally, however… I have heard glib responses such as “Every decision under uncertainty can be modeled as a bet” many times. Yet if the applicability of Dutch book theorems is so ubiquitous, why do you (and others who say similar things) seem to find it so difficult to provide an actual, concrete, real-world example of any of the claims in the OP? Not a class of examples; not an analogy; not even a formal proof that examples exist; but an actual example. In fact, it should not be onerous to provide—let’s say—three examples, yes? Please be specific.
If I cross the street, I make a bet about whether a car will run over me.
If I eat a pizza, I make a bet about whether the pizza will taste good.
If I’m posting this comment, I make a bet about whether it will convince anyone.
etc.
This one is not a central example, since I’ve not seen any VNM-proponent put it in quite these terms. A citation for this would be nice. In any case, the sort of thing you cite is not really my primary objection to VNM (insofar as I even have “objections” to the theorem itself rather than to the irresponsible way in which it’s often used), so we can let this pass.
VNM is used to show why you need to have utility functions if you don’t want to get Dutch-booked. It’s not something the OP invented, it’s the whole point of VNM. One wonder what you thought VNM was about.
Yes, this is exactly the claim under dispute. This is the one you need to be defending, seriously and in detail.
That we face trade-offs in the real world is a claim under dispute ?
Ditto.
Another way of phrasing it is that we can model “ignore” as a choice, and derive the VNM theorem just as usual.
Ditto again. I have asked for a demonstration of this claim many times, when I’ve seen Dutch Books brought up on Less Wrong and in related contexts. I’ve never gotten so much as a serious attempt at a response. I ask you the same: demonstrate, please, and with (real-world!) examples.
Ditto.
Once again, please provide some real-world examples of when this applies.
OP said it: every time we make a decision under uncertainty. Every decision under uncertainty can be modeled as a bet, and Dutch book theorems are derived as usual.
Is Aumann robust to untrustworthiness ?
This bidimensional model is weird.
I can imagine pure mania: assigning a 100% probability to everything going right
I can imagine pure depression: assign a 100% probability to everything going wrong
I can imagine pure anxiety: a completely flat probability distribution of things going right or wrong
But I can’t imagine pure top left mood. This lead me to think that the mood square is actually a mood triangle, and that there is no top left mood, only a spectrum of moods between anxiety and mania.
cough cough
At least, you do. (With apologies to Steven Brust)