@denis bider: ‘In the example you made, it appears as though you are using “superior” to mean “the one I like more” or “the one I think is worthy of praise” or “the one whose behavior should be encouraged”.’
I was using it as in “an actual is better than a potential.”
Having a high IQ doesn’t make someone a “superior human being” in my opinion, it’s what you do that counts. An man of average intelligence who starts a small business and employs some people is superior to a lazy genius.
Stephen: “the issue isn’t whether it could determine what humans want, but whether it would care.”
That is certainly an issue, but I think in this post and in Magical Categories, EY is leaving that aside for the moment, and simply focussing on whether we can hope to communicate what we want to the AI in the first place.
It seems to me that today’s computers are 100% literal and naive, and EY imagines a superintelligent computer still retaining that property, but would it?
Is intelligence general or not? If it is, then an entity that can do molecular engineering but be completely naive about what humans want it impossible.
People don’t believe that inanimate objects contain spirits any more but they do still believe that God can control such items which is almost the same thing. True rationality is realizing that they are not controlled by any mind—their own or God’s—but they do what they do because of what they are.
Caledonian—in matters of the heart perhaps people go with emotion, merely rationalizing after the fact, but in other areas—career, finances, etc, I think most people try to reason it out. You need to have more faith in your fellow man :)
The whole question of “should” only arises if you have a choice and a mind powerful enough to reason about it. If you watch dogs it does sometimes look like they are choosing. For example if two people call a dog simultaneously it will look here, look there, pause and think (it looks like) and then go for one of the people. But I doubt it has reasoned out the choice, it has probably just gone with whatever emotion strikes it at the key moment.
In the real world, everything worth having comes from someone’s effort—even wild fruit has to be picked, sorted, and cleaned and fish need to be caught, gutted etc. I think this universal fact of required effort is probably part of the data we get the concept of fairness from in the first place, so reasoning in a space where pies pop in to existence from nothing seems like whatever you conclude might not be applicable to the real world anyway.
In reality someone would have had to bake the pie, and it’s fair that they get it since they put in the work. The problem is that the author, in creating the example, eliminated certain facts such as the baker in order to get to the essence of the problem. But the more facts you eliminate the more chance that something will appear arbitrary, due to fewer paths back to reality. It’s the fallacy of the over-simplified model (no that’s not a real fallacy :).
It seems to me you’re saying that what our conscience tells us is right is right because it’s output is what we mean by “right” in the first place. While I agree in general that a concept is it’s referents, I don’t agree with what you’re saying here.
Those referents are not values but evaluations. And they are evaluations with respect to a standard that we can in fact change. We don’t choose the output of our conscience on the spot, in that sense it is objective, but over time we can reprogram it through repetition and effort. It’s evaluations are short-term objective but long term subjective.
The behavior of the objects in a counterfactual can be true to life, but the counterfactual as a whole can’t be true.
I also find it helpful to think that evolution works on timescales much greater than 1000 years, so the people that built that church tower were just as smart as us. i.e. they weren’t stupid savages, they were just low tech versions of us.
“Life is meaningless because it’s all just atoms in the void and nothing we do matters”
But, but … concepts must have referents to actually mean something, so to simultaneously say meaning is X, and by the way there’s none of it, does not compute.
If you take the list of things that were moral yesterday and the list that are moral today, and look for pairs between the lists that are kind of the same idea, but just in different quantity (e.g. like and love) then you could step back and see if there is an overall direction.
The key idea is to recognize when two things with different names are really different amounts of some higher more abstract idea.
Robin there’s no logic to trust. Even though we might use the phrase “therefore,” in reality we are not performing a process of reason, but just changing our perspective on a fact. Looking at the same data askance. That’s my way of thinking about it anyway.
I think Descartes was right. Even if this is the Matrix and everything is fake, even your thought processes, then it is all still something and not nothing. By nothing I mean a dimensionless point. So at the very least you are something. “I think therefore I am.”
There is a version of truth which says things are not true if they match reality, but if enough people believe it. Maybe that factors in to the dying friend thing also.
Surely the ultimate grounding for any argument has to be the facts of reality? Every chain of reasoning must end not by looping back on itself but by a hand pointing at something in reality.
I think the degree to which you can know tomorrow will be like today is determined by how well you understand the object in question, not by the number of past times it’s been that way.
For example you know your dog will bark and not quack tomorrow to the extent that you understand the workings of his voice box, not necessarily because he has barked the whole time you’ve known him (though that is evidence also). The more levels down you understand his voice box the surer you can be, because each level down eliminates more possibilities.
It’s deeper than science being only applicable to natural things—reason as such is only applicable to natural things. Once you are in the realm of the supernatural anything is possible and the laws of logic don’t necessarily hold. You have to just close your mouth and turn off your mind and have faith. Which does not give a teacher a lot of material to work with...