I would also like to affirm that thread’s claim that “if what you really want is ponies, the Truly Friendly AI will in fact give you ponies.” (“Really want”, of course, requires lots of unpacking.)
Not quite affirmable; a CEV-based FAI only gives you ponies if that’s what you would-want, if on average everyone would-want is to give a pony to someone who would-want one. (Because an individually based mechanism might e.g. look at babies and determine that what they would-want as individuals is eternal feeding and burping.)
Can you find a more obviously bad example of the implications of individually-based CEV? I find that if a baby would-want to feed and burp, that’s what it should get; and if two parents want to spawn another human with CEV rights, they might not want to spawn it initially only would-wanting to feed and burp.
Yes, though I find it improbable that they’d Really Want ponies.
(Devil’s advocate: there are people who participate in the fandom daily, and have big chunks of their identity tied up in being a brony. If there were actually a population where people would Really Want ponies, this would be the one.)
Wrote a cheque for $5,000.
(I put the redacted image of my donation online because someone else decided to start an ad-hoc fundraising effort for MIRI on FIMFiction.)
Thanks very much!
I would also like to affirm that thread’s claim that “if what you really want is ponies, the Truly Friendly AI will in fact give you ponies.” (“Really want”, of course, requires lots of unpacking.)
Not quite affirmable; a CEV-based FAI only gives you ponies if that’s what you would-want, if on average everyone would-want is to give a pony to someone who would-want one. (Because an individually based mechanism might e.g. look at babies and determine that what they would-want as individuals is eternal feeding and burping.)
Can you find a more obviously bad example of the implications of individually-based CEV? I find that if a baby would-want to feed and burp, that’s what it should get; and if two parents want to spawn another human with CEV rights, they might not want to spawn it initially only would-wanting to feed and burp.
Yes, though I find it improbable that they’d Really Want ponies.
(Devil’s advocate: there are people who participate in the fandom daily, and have big chunks of their identity tied up in being a brony. If there were actually a population where people would Really Want ponies, this would be the one.)