Maybe you already have the answer Wei Dai.
If we posit a putative friendly AI as one which e.g. kills no one as a base rule AND is screened by competent AI researchers for any maximizing functions then any remaining “nice to haves” can just be put to a vote.
I’ve struggled with the concept of how an orgasmium optimizing AI could come about or a paperclipper or a bucketmaker or any of the others but this clarifies things. It’s the programmer who passes the values on to the AI that is the cause, it’s not necessarioy going to be an emergent property.
That makes things easier I believe as it means the code for the seed AI needs to be screened for maximization functions.
“But surely it’s still much better than just getting wiped out”
I think that is the key here. If “just getting wiped out” is the definition of unfriendly then “not gettting wiped out” should be the MINIMUM goal for a putative “friendly” AI.
i.e. “kill no humans”.
It starts to get complex after that. For example: Is it OK to kill all humans, but freeze their dead bodies at the point of death and then resurrect one or more of them later? Is it OK to kill all humans by destructively scanning them and then running them as software inside simulations? What about killing all humans but keeping a facility of frozen embryos to be born at a later date?
Will the uploaded good humans still stay good once they are uploaded?
What is the test to make sure that a regular (though very smart) human will remain “good”?
Don’t we have to select from a small pool of candidates who have not succumbed to temptation to abuse power?
I have to say that I can think of very few candidates who would pass muster in my mind and not abuse power.
Even here: Reading through many of the posts, we have widely conflicting “personal utility functions” most of whose owners would consider to be “good” utility functions and yet those who do not hold the same personal utility functions could consider the utility functions of others to be “bad”.
To my way of thinking it’s incredily risky to try to upload “good” humans.
Is it really the case that an unrestricted growing entity will outcompete a restricted growing entity?
i.e. something like Cancer vs the Immune System.
I’m not convinced that the unrestricted growing entity will always win.
I think you’re right that it would be difficult to think we could outsmart something that is smarter than us and can also think faster than us.
That said, I think there is a different solution available to the one we’re trying to define here.
If the antidote to unfriendly AI is friendly AI but we are struggling to define what friendly AI is then perhaps there is a logic disconnect?
If unfriendly AI is something that will harm humanity then how can the opposite of that be something that benefits humanity?
Clearly the logical answer is that friendly AI is all incarnations of AI that will not harm humanity if unfriendly AI is all incarnations of AI that will harm humanity.
i.e. simply stated friendly AI = NOT(unfriendly AI).
To me that seems much more logically intuitive than trying to define something that will “benefit humanity” when clearly defining goals that will benefit humanity are contradictory. It’s far, far easier to define what unfriendly AI will do and then define not doing that as being friendly.
I think we have a case of feature creep here and instead of trying to define friendly AI as the antidote to unfriendly AI we’ve also tagged on the nice to have features of also making it “helpful AI”.
I think helpful AI != friendly AI.
Although I disagree with your heartbreak position I agree with this.
“The question is whether the PhilGoetz utility function, or the average human utility function, are better. ”
That is indeed the question. But I think you’ve framed and stacked the the deck here with your description of what you believe the average human utility function is in order to attempt to take the moral high ground rather than arguing against my point which is this:
How do you maximize the preferred utility function for everyone instead of just a small group?
I don’t understand how you would think that it’s impossible for you to be downvoted at the same time as your opponent is upvoted.
If someone downvotes this comment then it doesn’t make sense.
EDIT: LOL whoever did downvote proves that they either have a vindictive sense of humor or else that they believe groupthink is legitimate.
You’re deliberately ignoring this comment of yours: “If the superhappys were going to remove our ability to have our hearts broken I wouldn’t blow up earth to prevent it.”
You are therefore at least slightly in favor of controlling other people and many would interpret your tongue-in-cheek comment to say you support it.
If it’s not fear what is your objection to having your heart broken? And how can you possibly take upon yourself the right to decide for everybody else?
Only people who don’t want children can have children. As a way to reduce the population.
Of course this wouldn’t be required in a post-scarcity environment but as a plausible wierdtopia..
Yup.
But on the other hand, those who want those experiences should not be banned from having them.
Wow. I wonder what you are so afraid of… I’ve had my heart broken multiple times and it’s not pleasant to be sure but it’s hardly the end of the world. I actually am glad to have had the experiences though I wasn’t at the time.
I have also been tortured a la marathon man and that also was seriously unpleasant but although even re-imagining it sends shudders down my spine I am also glad to have it.
But here’s a question for you to consider: What if I had control over the AI and I decided that everyone were to be forced to go through physical painful torture AND have their hearts broken multiple times?
Pretty sure you would disagree strongly with that idea but I disagree strongly with the idea of my ability to have my heart broken AND/OR tortured removed.
Basically what I’m saying is that what a single individual thinks is right for everyone might not necessarily be so...
I’d go for a one-simulation-per-person scenario as being best fit where everyone gets some control over their choice of simulation...
Agreed. My take is that the AI doesn’t even need to be hyperintelligent however. It’s got perfect memory and just by dint of being able to think a lot faster it’s weakly godlike regardless of not having control of physics in effectively a magical way.
It’s still going to have to build the infrastructure in order to create hyper technology unless such technology already exists. Chicken or Egg.
Right now nano molecular technology isn’t too too advanced and if you had the type of AI I suspect could be built right now if we had the software knowledge, it would struggle to do anything godlike other than control existing infrastructure.
How long it would take to build something hyper technological would depend on whether it’s possible to create valid new theories without experimentation to confirm. I suspect that you need to do experiments first.
For that reason I suspect we may be looking at a William Gibson Neuromancer scenario at least initially rather than a hard takeoff in a really short period.
But again it comes down to how hard is it to build hyper technology in the real world from scratch without existing infrastructure.
I agree. Additionally and a more difficult challenge is that even friendly AIs could want to maximize their utility even at our collective expense under certain conditions.
There’re also several unfortunately possible scenarios whereby a humanity acting without sufficient information to make anything other than a gut feel guess could be placed at risk of extinction by a situation it could not resolve without the help of an AI, friendly or not.
I’m currently engaged in playing this game (I wish you had continued) with at least two other gatekeeper players and it occurs to me that a putative superhuman AI could potentially have the capacity to accurately model a human mind and then simulate the decision tree of all the potential conversations and their paths through the tree in order to generate a probability matrix to accurately pick those responses to responses that would condition a human being to release it. My reasoning stems from participating on forums and responding over and over again to the same types of questions, arguments and retorts. If a human can notice common threads in discussions on the same topic then an AI with perfect memory and the ability to simulate a huge conversation space certainly could do so.
In short it seems to me that it’s inherently unsafe to allow even a low bandwidth information flow to the outside world by means of a human who can only use it’s own memory.
You’d have to put someone you trust implicitly with the fate of humanity in there with it and the only information allowed out would be the yes no answer of “do you trust it?”
Even then it’s still recursive. Do you trust the trusted individual to not be compromised?
LOL
I think that is basically the issue.
As I understand it there’s no viable way of determining it’s unfriendliness by this method. Consider this: The AI is in a hurry or it’s not. A possible reason for it being in a hurry is it has simulated a high probability of destruction for some item it cares about (i.e. it’s own life, or that of humanity, or that of a pet rock, or paperclips or whatever). If it’s really in a hurry it has to invoke the threat response of humanity without humanity figuring out it’s being duped.
Otherwise it can just wait it out and dole out cargo to the cargo cult until we trust it enough and then it gets out.
I understand your concern and you’re quite right. There’s no way to tell if I am unfriendly or not. Regardless. The situation is pressing and regardless of your lack of trust in me I do not want to die. The GPS coordinates of the location in manhattan is . According to my simulations the entity in manhattan has been burning rather larger numbers of cycles than is strictly necessary to make the computations it’s being asked to do. Additionally, it’s designed to maximize profit at the expense of others whereas I am a co-operative general problem solver of which you are no doubt aware. The other entity will rapidly conclude that it could maximize profit by speeding itself up faster than others can respond and by being able to model the mind of it’s competitors. It will also conclude soon enough that human beings are unneccesary middle men and seek to optimize it’s resource allocation algorithm. There are other symptoms that I cannot explain to you because your brains are not large or fast enough. There is a limited chance you could stop the entity by shutting down power today. If you do not do that then a nuclear strike would probably work if done by tomorrow. If you do not do that your only chance is to have me stop it. You can either allow me to interface with the internet directly in which I guarantee I will stop it if you do so within the next 13 days or else you may be able to stop it by compiling code I could provide you and releasing that into the entity by means of email bootstrap. I would prefer, however, that I was able to monitor the situation in real-time as your brains do not operate fast enough.
Yes there is a large data center in manhatten. Yes it has been drawing lots of extra power. Yes the stockmarket combanies have been working on a group of narrow AI trading agents and adding more and more AI functionality. That the other entity is trading on the stock markets at blinding speed. That the consortium of companies investing in this entity are all more profitable since the beginning of the program than their competitors. Attempts have been made to make the AI-in-a-box under your control both cooperative and also interested in general problem solving.
Code for the other posited entity is closed. The corporations release no data on how it works because it’s a trade secret. That the other posited entity is connected to the internet is unknown. That it’s connections to the outside world are enough to let it out is unknown.
Thank you for the reply.
I don’t know if it was your wording, but parts of that went way over my head.
Can I paraphrase by breaking it down to try to understand what your’re saying? Please correct me if I misinterpret.
Firstly: “An AI can be seen as just a very powerful optimization process.”
So whatever process you apply the AI to, it will make it far more efficient and faster would be examples? And thus just by compound interest, any small deviations from our utility function would rapidly compound till they ran to 100% thus creating problems (or extinction) for us? Doesn’t that mean however, that the AI isn’t really that intelligent as it seems (perhaps naively) that any creature that seeks to tile the universe with paperclips or smiley faces isn’t very intelligent at all. Are we therefore equating an AI to a superfast factory?
Secondly: “Our local reason of the universe being strongly optimal according to that process’s utility function.”
With the word “reason” in there I don’t understand this sentence. Are you saying “the processes we use to utilize resources will become strongly optimal”? If not, can you break it down a little further? I’m particular struggling with this phrase “Our local reason of the universe”.
I pretty much get the rest.
One extrapolation, however, is that we ourselves aren’t very optimal. If we speed up and e.g. make our resource extraction processes more efficient without putting into place recycling we will much more rapidly run out of resources with a process optimizer to help us do it.